Keychain backend not working on Macs (v0.24.0)

[uvw]

Following up on #139 recent comments, Granted started failing after upgrading to v0.24.0 with the same error:

[✘] opening keyring: Specified keyring backend not available

Relevant logs:

{"level":"debug","ts":"2024-04-30T20:30:53Z","msg":"profile registry not configured. Skipping auto sync."}
{"level":"debug","ts":"2024-04-30T20:30:53Z","msg":"running credential process with config","profile":"backup","url":"","window":900,"disableCredentialProcessCache":false}
{"level":"debug","ts":"2024-04-30T20:30:53Z","msg":"error loading cached credentials","error":"opening keyring: Specified keyring backend not available","errorVerbose":"Specified keyring backend not available\nopening keyring\ngithub.com/common-fate/granted/pkg/securestorage.(*SecureStorage).openKeyring\n\t/Users/runner/work/granted-cli-build/granted-cli-build/granted/pkg/securestorage/securestorage.go:167\ngithub.com/common-fate/granted/pkg/securestorage.(*SecureStorage).Retrieve\n\t/Users/runner/work/granted-cli-build/granted-cli-build/granted/pkg/securestorage/securestorage.go:39\ngithub.com/common-fate/granted/pkg/securestorage.(*SessionCredentialSecureStorage).GetCredentials\n\t/Users/runner/work/granted-cli-build/granted-cli-build/granted/pkg/securestorage/session_credential_storage.go:28\ngithub.com/common-fate/granted/pkg/granted.init.func8\n\t/Users/runner/work/granted-cli-build/granted-cli-build/granted/pkg/granted/credential_process.go:52\ngithub.com/urfave/cli/v2.(*Command).Run\n\t/Users/runner/go/pkg/mod/github.com/urfave/cli/[email protected]/command.go:274\ngithub.com/urfave/cli/v2.(*Command).Run\n\t/Users/runner/go/pkg/mod/github.com/urfave/cli/[email protected]/command.go:267\ngithub.com/urfave/cli/v2.(*App).RunContext\n\t/Users/runner/go/pkg/mod/github.com/urfave/cli/[email protected]/app.go:332\ngithub.com/urfave/cli/v2.(*App).Run\n\t/Users/runner/go/pkg/mod/github.com/urfave/cli/[email protected]/app.go:309\nmain.main\n\t/Users/runner/work/granted-cli-build/granted-cli-build/granted/cmd/granted/main.go:50\nruntime.main\n\t/Users/runner/hostedtoolcache/go/1.22.1/arm64/src/runtime/proc.go:271\nruntime.goexit\n\t/Users/runner/hostedtoolcache/go/1.22.1/arm64/src/runtime/asm_amd64.s:1695"}
{"level":"debug","ts":"2024-04-30T20:30:53Z","msg":"error retrieving IAM Identity Center token from secure storage: opening keyring: Specified keyring backend not available"}
{"level":"debug","ts":"2024-04-30T20:30:53Z","msg":"writing sso token to credentials cache: opening keyring: Specified keyring backend not available"}
{"level":"debug","ts":"2024-04-30T20:30:54Z","msg":"storing refreshed credentials in credential process cache","expires":"2024-04-30 21:30:53 -0700 PDT","canExpire":true,"timeNow":"2024-04-30 13:30:54.400102 -0700 PDT m=+0.519999061"}

.granted/config:

DefaultBrowser = "CHROME"
CustomBrowserPath = "/Applications/Google Chrome.app/Contents/MacOS/Google Chrome"
CustomSSOBrowserPath = ""
Ordering = "Frecency"
ExportCredentialSuffix = ""

[Keyring]
  Backend = "keychain"

However, in my case, it is an old Intel x86_64 MacBook Pro.

Rolling back to v0.23.2 fixed the issue.

I didn't find anything related to keyring or securestorage in the v0.24.0 changes (v0.23.2...v0.24.0), but it seems like a regression.

[JoshuaWilkes]

Hi @uvw

I have tested out the latest release on my mac m3 pro and it appears to be working as expected.
I'm wondering if this is something related to our build environment, given that the keychain package has not been updated in some time.

Some steps that would help us to diagnose the issue further would be to try building from source on your machine.

If possible could you follow the steps in the contributing guide to build the cli then run dassume to test

[citosid]

@JoshuaWilkes, I tried building locally and it works as expected:

[acruz@blinkin] ~/github/granted (main)
❯ dassume --verbose
[DEBUG] profile registry not configured. Skipping auto sync.
[DEBUG] process args    execFlag:       osargs:[dassumego,--verbose]    c.args:[]
[DEBUG] processed profile name
[DEBUG] exec config:<nil>
[DEBUG] skipping profile with name my-account/Administrator - profile already defined in config

? Please select the profile you would like to assume: my-account/Administrator
[i] To assume this profile again later without needing to select it, run this command:
> assume my-account/Administrator --verbose
2024/05/01 08:54:46 [keyring] Considering backends: [keychain]
2024/05/01 08:54:46 [keyring] Querying keychain for service="granted-aws-sso-tokens", account="https://xxxx.awsxxxx.com/start", keychain="login.keychain"
2024/05/01 08:54:47 [keyring] No results found
[DEBUG] error retrieving IAM Identity Center token from secure storage: The specified item could not be found in the keyring
[i] If the browser does not open automatically, please open this link: https://device.sso.us-east-1.amazonaws.com/?user_code=JDKN-GLGK
[i] Awaiting AWS authentication in the browser
[i] You will be prompted to authenticate with AWS in the browser, then you will be prompted to 'Allow'
[i] Code: JDKN-GLGK
2024/05/01 08:54:54 [keyring] Considering backends: [keychain]
2024/05/01 08:54:54 [keyring] Checking keychain status
2024/05/01 08:54:54 [keyring] Keychain status returned nil, keychain exists
2024/05/01 08:54:54 [keyring] Keychain item trusts keyring
2024/05/01 08:54:54 [keyring] Adding service="granted-aws-sso-tokens", label="", account="https://xxxx.awsxxxx.com/start", trusted=true to osx keychain "login.keychain"
[✔] [my-account/Administrator](us-east-1) session credentials will expire in 12 hours

[acruz@blinkin] ~/github/granted (main)

[citosid]

Another update. Running make cli-act-prod and allowing the binary to access the keychain fixes the issue as well.

So, in the meantime I'll be doing it like this. Hopefully a solution can be found for this problem. Sorry I cannot help debugging more... not really sure how to even replicate it.

[uvw]

Hi @JoshuaWilkes, thank you for looking into this.

I agree, it's not related to the code changes. But I don't think the build environment is involved either. It's rather Homebrew + keychain or just keychain access. Here is what I tried to isolate the problem:

1. Roll back to v0.23.2 in Homebrew: The first thing I tried, and it worked.

2. Build v0.24.0 from the sources: I did something similar to @citosid but performed all the steps manually:

go build -o ./bin/dgranted cmd/granted/main.go
ln -s $(pwd)/bin/dgranted ~/.local/bin
dgranted credential-process --profile <PROFILE> --auto-login
# {"Version":1,"AccessKeyId":"...","SecretAccessKey": "...", ...}

Running dgranted credential-process asked for access to the keychain and worked as expected. No errors or warnings in ~/.dgranted/log.

3. Use v0.24.0 binaries from the release:

curl -OL https://releases.commonfate.io/granted/v0.24.0/granted_0.24.0_darwin_x86_64.tar.gz
mkdir granted-bin
tar -xzf granted_0.24.0_darwin_x86_64.tar.gz -C granted-bin
ln -s $(pwd)/granted-bin/granted ~/.local/bin
granted credential-process --profile <PROFILE> --auto-login
# {"Version":1,"AccessKeyId":"...","SecretAccessKey": "...", ...}

Running granted credential-process asked for access to the keychain and worked as expected. No errors or warnings in ~/.granted/log. So the issue is not with the release binaries.

However, when I tried granted credential-process using v0.24.0 installed via Homebrew, it did not prompt for the keychain access and just failed with the "opening keyring" error.

Let me know if you'd like me to do more testing. I'm unfamiliar with how keychain works, so you might have to guide me.

[JoshuaWilkes]

Thanks for you help in isolating the root cause here, we have a few threads going at the moment discussing this in our community slack as well.

We isolated the likely cause to a change in our release pipeline and we are currently working to identify what the cause is.

[chrnorm]

Hi all, we've set up a prerelease build pipeline to help diagnose the issue. If you're affected by this and could help test a new prerelease build that would be fantastic. This new build has CGO_ENABLED=1 set in the release process which I think could be part of the issue:

curl -OL https://releases.commonfate.io/granted/granted_0.24.1-prerelease_darwin_x86_64.tar.gz
sudo tar -zxvf ./granted_0.24.1_darwin_x86_64.tar.gz -C /usr/local/bin/
ln -s /usr/local/bin/granted /usr/local/bin/assumego

If you're testing from a brew version you may need to uninstall it with brew uninstall granted. You can check which version you're running with which granted - the path should be /usr/local/bin/granted for the prerelease.

[citosid]

Thanks @chrnorm! Will try it later today and report back

[uvw]

Great! I switched to this build and so far so good. Thank you, @chrnorm 👍

I also cleared all granted-* keychain items and CLI SSO cache beforehand to start from the blank slate.

[chrnorm]

Great, thanks for all your help here @uvw @citosid! I'll close this as fixed in https://github.com/common-fate/granted/releases/tag/v0.25.0 which is being released now, but please reopen the issue if you encounter this again.

[citosid]

Thanks for your hard work @chrnorm ! It works now as expected.