SPM unknown error reference not found when changing branch

Question

Some of my project's dependencies are hosted on a private repository. It works most of the time, but sometimes when I change the current branch with git, I get this error :

❌ git@my_private_repo.git: An unknown error occurred. reference 'refs/remotes/origin/main' not found (-1)

And from that point, it's impossible to compile and my only option is to reset the SPM cache which takes a lot of time.

Any idea on what causes this and how to solve it?

Answer 1

Like others have answered, the issue can be worked around by deleting Swift package caches both in the derived data directory of your project, as well as the global Swift Package Manager cache in ~/Library/Caches/org.swift.swiftpm. Even better, by deleting only the affected remotes directory, but it can be time consuming to find the package and the folder.

Here is a small script I quickly put together that will delete all directories named remotes both in the global Swift Package Manager cache and in the derived data repositories directory of your project.

#!/bin/bash

if [[ $# -eq 0 ]] ; then
    echo 'Please call the script with the name of your project as it appears in the derived data directory. Case-insensitive.'
    echo 'For example: ./fix-spm-cache.sh myproject'
    exit 0
fi

# Delete all directories named "remotes" from the global Swift Package Manager cache.
cd ~/Library/Caches/org.swift.swiftpm/repositories

for i in $(find . -name "remotes" -type d); do
    echo "Deleting $i"
    rm -rf $i
done

# Find derived data directories for all projects matching the script argument, and
# delete all directories named "remotes" from source package repositories cache for those projects. 

cd ~/Library/Developer/Xcode/DerivedData/

for project in $(find . -iname "$1*" -type d -maxdepth 1); do
    for i in $(find "$project/SourcePackages/repositories" -name "remotes" -type d); do
        echo "Deleting $i"
        rm -rf $i
    done
done

If you save the code in a file named fix-spm-cache.sh, then you can do chmod +x fix-spm-cache.sh to make the file executable. Afterwards, just run the script with the project name, like ./fix-spm-cache.sh myproject, when you encounter the error in Xcode.

You can keep Xcode open while running the script. Once the script finishes executing, try resolving or updating your packages again. It should work, and it should be relatively fast since we are not deleting the whole cache.

This should resolve both the error mentioned in this topic, as well as the infamous SwiftPM.SPMRepositoryError error 5 that happens in Xcode 13, which is probably the same error, just with a different message.

Answer 2

I think I may have found the issue here! After a ton of digging it seems Xcode and Swift PM has a bug with repos using git@ rather than https://

Using ssh we are getting a hanging ref to remote/origin/main in our caches and derived data. When using https this is not the case. This makes sense as the only dep in our project using ssh are our internal deps.

I tested this by adding a brand new 3rd party dependency git@github.com:jpsim/Yams.git into our project and saw the cache in org.swift.swiftpm update incorrectly.

UPDATE: 10 days later and it seems this was the issue all along. However even after changing the references it took a full DerivedData/SPM cache/Package.resolved wipe before Xcode no longer had any references to the git@ repos. I've filed a feedback with Apple for this bug.

Answer 3

After having this problem in XCode and AppCode, which does not have a "Reset Package Cache" button and takes forever re-indexing the whole iOS Framework when you delete the DerivedData folder, i tried to minimize the folders i needed to delete.

I ended up with the following: Once you get the error message git@library-project.git: An unknown error occurred. reference 'refs/remotes/origin/main' not found (-1), you

1. go to your DerivedData folder (~/Library/Developer/Xcode/DerivedData/) (for AppCode, it's located in ~/Library/Caches/JetBrains/AppCode2022.2/DerivedData)
2. go to the following directory, matching your project name, of course: <YourProject>-<RandomCharacters>/SourcePackages/repositories/<FailingLibraryProject>-<RandomCharacters>/refs/
3. delete the remotes folder
4. In XCode, click File -> Packages -> Resolve Package Versions
5. In AppCode, click Tools -> Swift Package Manager -> Resolve Dependencies.
6. Close and Re-open XCode

That's all it takes - at least for me. I guess you need to repeat the steps for every library project that's failing for you. And the best thing: I never need to do this again, no matter how many times i switch branches or dependency versions :)

Answer 4

Had same issue, even created new repo with same content to see that it is working. In the end I found a solution that helped me.

1. Remove package from project
2. Close Xcode
3. Clean SPM - swift package purge-cache
4. remove DerivedData - rm -rf ~/Library/Developer/Xcode/DerivedData
5. Open project and add package again.

P.S. purge-cache w/o removing DerivedData not worked, but it could be that only removing DerivedData was required to solve this issue. Can not re-check as I can not reproduce this issue anymore.

UPD: Step #3 not required.

Answer 5

Something that seemed to work for me today:

• Temporarily remove the dependency that causes the issue
• Close Xcode
• Checkout the Package.resolved to cancel its changes
• Reopen Xcode

After a few seconds, everything seemed to be normal again.

Answer 6

I made a python script to fix the issue. The idea is to get the latest commit hash and write it to Package.resolved, then resolvePackageDependencies for you.

You can get the script below, or download from here

# Sometimes Xcode cannot resolve SPM(File -> Packages -> Resolve Package versions) if the dependency url is ssh
# This script is a workaround to resolve package versions.
# Usage:
#     python spmResolve.py
# or
#     python3 spmResolve.py
import os.path
import subprocess
import glob
import json


def main():
    package_file = "xcshareddata/swiftpm/Package.resolved"
    xcodeproj = glob.glob('*.xcodeproj')
    xcworkspace = glob.glob('*.xcworkspace')
    spmproj = glob.glob('Package.resolved')
    package_resolved = ""
    if xcodeproj:
        package_resolved = xcodeproj[0] + f"/project.xcworkspace/{package_file}"
    elif xcworkspace:
        package_resolved = xcworkspace[0] + f"/{package_file}"
    elif spmproj:
        package_resolved = spmproj[0]
    else:
        print(f" Cannot find *.xcodeproj, *.xcworkspace or Package.resolved file")
        exit(-1)

    update_package_resolved(package_resolved)


def update_package_resolved(package_resolved):
    if not os.path.exists(package_resolved):
        print(f" Package.resolved file doesn't exit: {package_resolved}")
        exit(-1)

    print(f"Found: {package_resolved}")
    f = open(package_resolved)
    content = json.load(f)
    f.close()
    for pin in content["pins"]:
        url = pin["location"]
        if "branch" in pin["state"]:
            branch = pin["state"]["branch"]
            commit_hash = get_git_revision_hash(url, branch)
            print(f"{url}, {branch}, {commit_hash}")
            pin["state"]["revision"] = commit_hash
        elif "version" in pin["state"]:
            version = pin["state"]["version"]
            commit_hash = get_git_revision_by_tag(url, version)
            print(f"{url}, {version}, {commit_hash}")
            pin["state"]["revision"] = commit_hash

    with open(package_resolved, "w") as output:
        json.dump(content,  output, indent=4)

    # resolve SPM
    subprocess.run(['xcodebuild', '-resolvePackageDependencies'])
    print(' Well done')


def get_git_revision_hash(url, branch) -> str:
    command = f'git ls-remote {url} refs/heads/{branch} | cut -f 1'
    return get_git_command_output(command)


def get_git_revision_by_tag(url, version) -> str:
    command = f'git ls-remote {url} -t {version} | cut -f 1'
    return get_git_command_output(command)


def get_git_command_output(command) -> str:
    return subprocess.check_output(command, stderr=subprocess.STDOUT, shell=True).decode('ascii').rstrip()


if __name__ == '__main__':
    main()

Put the script in your xcode project, along with your *.xcodeproj or *.xcworkspace. Then run

python spmResolve.py

or

python3 spmResolve.py

If you don't have python:

brew install python

Answer 7

I had the same error but with repo added via https. I just removed cache at ~/Library/Caches/org.swift.swiftpm and seems like it works. Reset packages, clear derived data didn't help.