Keychain backend not working on macOS

[mlangenberg]

Describe the bug
The keychain backend is not working on macOS.

To Reproduce
Steps to reproduce the behavior:

1. brew install gphotosuploader/tap/gphotos-uploader-cli
2. Run gphotos-uploader-cli init
3. Edit config.hjson and set SecretsBackendType to keychain.
4. Run gphotos-uploader-cli push

Expected behavior
Be able to run the program without further user interaction whilst having the Google Photos credentials safely stored in the macOS keychain.

Actual behavior

gphotos-uploader-cli push
[info]   Reading configuration from '~/.gphotos-uploader-cli/config.hjson'
[error]  Token manager could not be started, err: Specified keyring backend not available
[fatal]  token manager could not be started, type:keychain, err: Specified keyring backend not available

Desktop & Client versions (please complete the following information):

• Operating System: macOS Catalina 10.15.7
• Version gphotos-uploader-cli v3.1.0

[mlangenberg]

As a temporary workaround, you can add the passphrase manually to the macOS keychain.

Then set GPHOTOS_CLI_TOKENSTORE_KEY environment variable, with the output of security find-generic-password -a $USER -s name_of_item -w in a wrapper script before calling gphotos-uploader-cli push.

[pacoorozco]

Hi @mlangenberg

First of all, thanks for submitting the bug. To be honest, I have not tested the macOS keychain integration. gphotos-uploader-cli uses the awesome 99designs/keyring library for keeping the secrets safe.

The 99designs/keyring library documentation is pointing to several configuration keys that are probably missing in your setup.

I have to figure out what config items are missing.

[mlangenberg]

Okay, let me know if I can test something for you. And thanks for the effort in protecting the credentials to my Google Photos library.

[pacoorozco]

Hi @mlangenberg
I've been doing digging in the code and there is something that I can't understand. I would expect a message asking for a password, something like:

$ gphotos-uploader-cli push
[info]   Reading configuration from '~/.gphotos-uploader-cli/config.hjson'
Enter the passphrase to open the token store: 

But I haven't seen it on your issue. Could you run again the push command using the debug flag on a terminal?

$ gphotos-uploader-cli push --debug

[mlangenberg]

Sure, here is the output.

$ gphotos-uploader-cli --config /tmp/gphotos-uploader-cli init
[info]   Using application data at '/tmp/gphotos-uploader-cli'.
[done] √ Application data dir created successfully.

Please edit:
- `/tmp/gphotos-uploader-cli/config.hjson` to add your configuration.

$ gphotos-uploader-cli --config /tmp/gphotos-uploader-cli --debug push
[info]   Reading configuration from '/tmp/gphotos-uploader-cli/config.hjson'
[debug]  Current configuration: {"APIAppCredentials":{"ClientID":"YOUR_APP_CLIENT_ID","ClientSecret":"REMOVED"},"Account":"[email protected]","SecretsBackendType":"keychain","Jobs":[{"SourceFolder":"/tmp/photos-test","CreateAlbums":"folderName","DeleteAfterUpload":false,"IncludePatterns":[],"ExcludePatterns":[]}]}
[error]  Token manager could not be started, err: Specified keyring backend not available
[fatal]  token manager could not be started, type:keychain, err: Specified keyring backend not available

[mlangenberg]

I also compiled from source (a1ba508), to see if that made any difference. This gives a more useful error message, so it partly has to do with the homebrew build. Although keychain support still does not work.

./gphotos-uploader-cli --config /tmp/gphotos-uploader-cli --debug push
[info]   Reading configuration from '/tmp/gphotos-uploader-cli/config.hjson'
[debug]  Current configuration: {"APIAppCredentials":{"ClientID":"XXX","ClientSecret":"REMOVED"},"Account":"[email protected]","SecretsBackendType":"keychain","Jobs":[{"SourceFolder":"/Users/user/Pictures/gphotos-test","CreateAlbums":"folderName","DeleteAfterUpload":false,"IncludePatterns":[],"ExcludePatterns":[]}]}
[info]   Getting OAuth token for '[email protected]'
[debug]  Unable to retrieve token from token manager: token was not found
[debug]  Getting OAuth2 token from prompt...

Visit the following URL in your browser:
https://accounts.google.com/o/oauth2/auth?access_type=offline&client_id=XXX&redirect_uri=urn%3Aietf%3Awg%3Aoauth%3A2.0%3Aoob&response_type=code&scope=https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fphotoslibrary&state=state

After completing the authorization flow, enter the authorization code here: ABC
[done] √ Token is valid, expires at 2021-09-08 17:35:45.713844 +0200 CEST m=+3643.884058254
Enter the passphrase to open the token store: fatal error: unexpected signal during runtime execution
[signal SIGSEGV: segmentation violation code=0x1 addr=0xb01dfacedebac1e pc=0x7fff7154370a]

Full stack trace

./gphotos-uploader-cli --config /tmp/gphotos-uploader-cli --debug push
[info]   Reading configuration from '/tmp/gphotos-uploader-cli/config.hjson'
[debug]  Current configuration: {"APIAppCredentials":{"ClientID":"XXX","ClientSecret":"REMOVED"},"Account":"[email protected]","SecretsBackendType":"keychain","Jobs":[{"SourceFolder":"/Users/user/Pictures/gphotos-test","CreateAlbums":"folderName","DeleteAfterUpload":false,"IncludePatterns":[],"ExcludePatterns":[]}]}
[info]   Getting OAuth token for '[email protected]'
[debug]  Unable to retrieve token from token manager: token was not found
[debug]  Getting OAuth2 token from prompt...

Visit the following URL in your browser:
https://accounts.google.com/o/oauth2/auth?access_type=offline&client_id=XXX&redirect_uri=urn%3Aietf%3Awg%3Aoauth%3A2.0%3Aoob&response_type=code&scope=https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fphotoslibrary&state=state

After completing the authorization flow, enter the authorization code here: ABC
[done] √ Token is valid, expires at 2021-09-08 17:35:45.713844 +0200 CEST m=+3643.884058254
Enter the passphrase to open the token store: fatal error: unexpected signal during runtime execution
[signal SIGSEGV: segmentation violation code=0x1 addr=0xb01dfacedebac1e pc=0x7fff7154370a]

runtime stack:
runtime: unexpected return pc for runtime.sigpanic called from 0x7fff7154370a
stack: frame={sp:0x7ffeefbff858, fp:0x7ffeefbff8a8} stack=[0x7ffeefb808f8,0x7ffeefbff960)
0x00007ffeefbff758:  0x01007ffeefbff778  0x0000000000000004
0x00007ffeefbff768:  0x000000000000001f  0x00007fff7154370a
0x00007ffeefbff778:  0x0b01dfacedebac1e  0x0000000000000001
0x00007ffeefbff788:  0x00000000040370b1 <runtime.throw+0x0000000000000071>  0x00007ffeefbff828
0x00007ffeefbff798:  0x0000000004492aae  0x00007ffeefbff7e0
0x00007ffeefbff7a8:  0x0000000004037368 <runtime.fatalthrow.func1+0x0000000000000048>  0x00000000048048e0
0x00007ffeefbff7b8:  0x0000000000000001  0x0000000000000001
0x00007ffeefbff7c8:  0x00007ffeefbff828  0x00000000040370b1 <runtime.throw+0x0000000000000071>
0x00007ffeefbff7d8:  0x00000000048048e0  0x00007ffeefbff818
0x00007ffeefbff7e8:  0x00000000040372f0 <runtime.fatalthrow+0x0000000000000050>  0x00007ffeefbff7f8
0x00007ffeefbff7f8:  0x0000000004037320 <runtime.fatalthrow.func1+0x0000000000000000>  0x00000000048048e0
0x00007ffeefbff808:  0x00000000040370b1 <runtime.throw+0x0000000000000071>  0x00007ffeefbff828
0x00007ffeefbff818:  0x00007ffeefbff848  0x00000000040370b1 <runtime.throw+0x0000000000000071>
0x00007ffeefbff828:  0x00007ffeefbff830  0x00000000040370e0 <runtime.throw.func1+0x0000000000000000>
0x00007ffeefbff838:  0x0000000004499331  0x000000000000002a
0x00007ffeefbff848:  0x00007ffeefbff898  0x000000000404c856 <runtime.sigpanic+0x0000000000000396>
0x00007ffeefbff858: <0x0000000004499331  0x00007ffeefbff880
0x00007ffeefbff868:  0xffffffffff810780  0xfffffffeffffffff
0x00007ffeefbff878:  0x0000000000000043  0x00007ffeefbff8e0
0x00007ffeefbff888:  0x00007fff37550167  0x000000c001106418
0x00007ffeefbff898:  0x00007ffeefbff8e0 !0x00007fff7154370a
0x00007ffeefbff8a8: >0x00007ffeefbff8e0  0x00000000047a8000
0x00007ffeefbff8b8:  0x00000000000007f1  0x000000000435dd85 <golang.org/x/sys/unix.libc_ioctl_trampoline+0x0000000000000005>
0x00007ffeefbff8c8:  0x0000000004068e7f <runtime.syscall+0x000000000000001f>  0x000000c00016cc00
0x00007ffeefbff8d8:  0x0000000000000043  0x000000c00016cbd0
0x00007ffeefbff8e8:  0x0000000004066cf0 <runtime.asmcgocall+0x0000000000000070>  0x0000000000000001
0x00007ffeefbff8f8:  0x0000000004012d00 <runtime.mapdelete+0x0000000000000020>  0x2d00000000000020
0x00007ffeefbff908:  0x00000000048048e0  0x0000000004838480
0x00007ffeefbff918:  0x0000000000001458  0x000000c0000001a0
0x00007ffeefbff928:  0x0000000004064e09 <runtime.systemstack+0x0000000000000049>  0x0000000000000004
0x00007ffeefbff938:  0x0000000004512000  0x00000000048048e0
0x00007ffeefbff948:  0x00007ffeefbff990  0x0000000004064d05 <runtime.mstart+0x0000000000000005>
0x00007ffeefbff958:  0x0000000004064cbd <runtime.rt0_go+0x000000000000013d>
runtime.throw({0x4499331, 0x7ffeefbff880})
	/Users/user/local/homebrew/Cellar/go/1.17/libexec/src/runtime/panic.go:1198 +0x71
runtime: unexpected return pc for runtime.sigpanic called from 0x7fff7154370a
stack: frame={sp:0x7ffeefbff858, fp:0x7ffeefbff8a8} stack=[0x7ffeefb808f8,0x7ffeefbff960)
0x00007ffeefbff758:  0x01007ffeefbff778  0x0000000000000004
0x00007ffeefbff768:  0x000000000000001f  0x00007fff7154370a
0x00007ffeefbff778:  0x0b01dfacedebac1e  0x0000000000000001
0x00007ffeefbff788:  0x00000000040370b1 <runtime.throw+0x0000000000000071>  0x00007ffeefbff828
0x00007ffeefbff798:  0x0000000004492aae  0x00007ffeefbff7e0
0x00007ffeefbff7a8:  0x0000000004037368 <runtime.fatalthrow.func1+0x0000000000000048>  0x00000000048048e0
0x00007ffeefbff7b8:  0x0000000000000001  0x0000000000000001
0x00007ffeefbff7c8:  0x00007ffeefbff828  0x00000000040370b1 <runtime.throw+0x0000000000000071>
0x00007ffeefbff7d8:  0x00000000048048e0  0x00007ffeefbff818
0x00007ffeefbff7e8:  0x00000000040372f0 <runtime.fatalthrow+0x0000000000000050>  0x00007ffeefbff7f8
0x00007ffeefbff7f8:  0x0000000004037320 <runtime.fatalthrow.func1+0x0000000000000000>  0x00000000048048e0
0x00007ffeefbff808:  0x00000000040370b1 <runtime.throw+0x0000000000000071>  0x00007ffeefbff828
0x00007ffeefbff818:  0x00007ffeefbff848  0x00000000040370b1 <runtime.throw+0x0000000000000071>
0x00007ffeefbff828:  0x00007ffeefbff830  0x00000000040370e0 <runtime.throw.func1+0x0000000000000000>
0x00007ffeefbff838:  0x0000000004499331  0x000000000000002a
0x00007ffeefbff848:  0x00007ffeefbff898  0x000000000404c856 <runtime.sigpanic+0x0000000000000396>
0x00007ffeefbff858: <0x0000000004499331  0x00007ffeefbff880
0x00007ffeefbff868:  0xffffffffff810780  0xfffffffeffffffff
0x00007ffeefbff878:  0x0000000000000043  0x00007ffeefbff8e0
0x00007ffeefbff888:  0x00007fff37550167  0x000000c001106418
0x00007ffeefbff898:  0x00007ffeefbff8e0 !0x00007fff7154370a
0x00007ffeefbff8a8: >0x00007ffeefbff8e0  0x00000000047a8000
0x00007ffeefbff8b8:  0x00000000000007f1  0x000000000435dd85 <golang.org/x/sys/unix.libc_ioctl_trampoline+0x0000000000000005>
0x00007ffeefbff8c8:  0x0000000004068e7f <runtime.syscall+0x000000000000001f>  0x000000c00016cc00
0x00007ffeefbff8d8:  0x0000000000000043  0x000000c00016cbd0
0x00007ffeefbff8e8:  0x0000000004066cf0 <runtime.asmcgocall+0x0000000000000070>  0x0000000000000001
0x00007ffeefbff8f8:  0x0000000004012d00 <runtime.mapdelete+0x0000000000000020>  0x2d00000000000020
0x00007ffeefbff908:  0x00000000048048e0  0x0000000004838480
0x00007ffeefbff918:  0x0000000000001458  0x000000c0000001a0
0x00007ffeefbff928:  0x0000000004064e09 <runtime.systemstack+0x0000000000000049>  0x0000000000000004
0x00007ffeefbff938:  0x0000000004512000  0x00000000048048e0
0x00007ffeefbff948:  0x00007ffeefbff990  0x0000000004064d05 <runtime.mstart+0x0000000000000005>
0x00007ffeefbff958:  0x0000000004064cbd <runtime.rt0_go+0x000000000000013d>
runtime.sigpanic()
	/Users/user/local/homebrew/Cellar/go/1.17/libexec/src/runtime/signal_unix.go:719 +0x396

goroutine 1 [syscall]:
syscall.syscall(0x435dd80, 0x0, 0x40487413, 0xc00016cc98)
	/Users/user/local/homebrew/Cellar/go/1.17/libexec/src/runtime/sys_darwin.go:22 +0x3b fp=0xc00016cc00 sp=0xc00016cbe0 pc=0x406393b
syscall.syscall(0x80000000000, 0xc00016cc68, 0x40986a6, 0xc0000780c0)
	<autogenerated>:1 +0x26 fp=0xc00016cc48 sp=0xc00016cc00 pc=0x4069686
golang.org/x/sys/unix.ioctl(0xc0000780c0, 0xc000174000, 0x40687e0)
	/Users/user/go/pkg/mod/golang.org/x/[email protected]/unix/zsyscall_darwin_amd64.go:689 +0x39 fp=0xc00016cc78 sp=0xc00016cc48 pc=0x435d8d9
golang.org/x/sys/unix.IoctlGetTermios(...)
	/Users/user/go/pkg/mod/golang.org/x/[email protected]/unix/ioctl.go:72
golang.org/x/term.readPassword(0x0)
	/Users/user/go/pkg/mod/golang.org/x/[email protected]/term_unix.go:75 +0xa5 fp=0xc00016cd88 sp=0xc00016cc78 pc=0x435df85
golang.org/x/term.ReadPassword(...)
	/Users/user/go/pkg/mod/golang.org/x/[email protected]/term.go:57
github.com/gphotosuploader/gphotos-uploader-cli/internal/datastore/tokenmanager.(*stdInPasswordReader).ReadPassword(0x4519840)
	/Users/user/dev/vendor/gphotos-uploader-cli/internal/datastore/tokenmanager/keyring_repository.go:116 +0x25 fp=0xc00016cde0 sp=0xc00016cd88 pc=0x4364585
github.com/gphotosuploader/gphotos-uploader-cli/internal/datastore/tokenmanager.promptFn.func1({0xc0000261b0, 0x67})
	/Users/user/dev/vendor/gphotos-uploader-cli/internal/datastore/tokenmanager/keyring_repository.go:108 +0xa2 fp=0xc00016ce38 sp=0xc00016cde0 pc=0x4364542
github.com/99designs/keyring.(*keychain).createOrOpen(0xc0001a66f0)
	/Users/user/go/pkg/mod/github.com/99designs/[email protected]/keychain.go:288 +0x185 fp=0xc00016cf70 sp=0xc00016ce38 pc=0x4362225
github.com/99designs/keyring.(*keychain).Set(0xc0001a66f0, {{0xc0000264c8, 0x18}, {0xc00116e000, 0x172, 0x180}, {0x0, 0x0}, {0x0, 0x0}, ...})
	/Users/user/go/pkg/mod/github.com/99designs/[email protected]/keychain.go:151 +0x45 fp=0xc00016d258 sp=0xc00016cf70 pc=0x4360a65
github.com/gphotosuploader/gphotos-uploader-cli/internal/datastore/tokenmanager.(*KeyringRepository).Set(0xc000063320, {0xc0000264c8, 0x18}, 0xc00016d318)
	/Users/user/dev/vendor/gphotos-uploader-cli/internal/datastore/tokenmanager/keyring_repository.go:66 +0xf5 fp=0xc00016d310 sp=0xc00016d258 pc=0x43641d5
github.com/gphotosuploader/gphotos-uploader-cli/internal/datastore/tokenmanager.(*TokenManager).Put(0x47f82c0, {0xc0000264c8, 0x1d}, 0xc00112b140)
	/Users/user/dev/vendor/gphotos-uploader-cli/internal/datastore/tokenmanager/tokenmanager.go:47 +0xad fp=0xc00016d340 sp=0xc00016d310 pc=0x43646ad
github.com/gphotosuploader/gphotos-uploader-cli/internal/app.App.NewOAuth2Client({{0x45237d8, 0xc00006c040}, {0x45202f8, 0xc000063330}, {0x4523848, 0xc000620050}, 0x0, {0x452df08, 0x47f82c0}, {0x452b398, ...}, ...}, ...)
	/Users/user/dev/vendor/gphotos-uploader-cli/internal/app/oauth.go:66 +0x752 fp=0xc00016d4c0 sp=0xc00016d340 pc=0x43741f2
github.com/gphotosuploader/gphotos-uploader-cli/internal/app.Start({0x4523570, 0xc000024098}, {0x7ffeefbffb40, 0x19})
	/Users/user/dev/vendor/gphotos-uploader-cli/internal/app/app.go:72 +0x591 fp=0xc00016d840 sp=0xc00016d4c0 pc=0x4372b91
github.com/gphotosuploader/gphotos-uploader-cli/internal/cmd.(*PushCmd).Run(0xc00000e228, 0xc00016dd88, {0xc00016dd60, 0x0, 0x0})
	/Users/user/dev/vendor/gphotos-uploader-cli/internal/cmd/push.go:50 +0x71 fp=0xc00016dd78 sp=0xc00016d840 pc=0x43ba5b1
github.com/gphotosuploader/gphotos-uploader-cli/internal/cmd.(*PushCmd).Run-fm(0xc000198c80, {0xc0001a6480, 0x3, 0x3})
	/Users/user/dev/vendor/gphotos-uploader-cli/internal/cmd/push.go:48 +0x3a fp=0xc00016ddb0 sp=0xc00016dd78 pc=0x43bc27a
github.com/spf13/cobra.(*Command).execute(0xc000198c80, {0xc0001a63f0, 0x3, 0x3})
	/Users/user/go/pkg/mod/github.com/spf13/[email protected]/command.go:826 +0x60e fp=0xc00016de70 sp=0xc00016ddb0 pc=0x418fece
github.com/spf13/cobra.(*Command).ExecuteC(0x47fe200)
	/Users/user/go/pkg/mod/github.com/spf13/[email protected]/command.go:914 +0x2fc fp=0xc00016df28 sp=0xc00016de70 pc=0x41903fc
github.com/spf13/cobra.(*Command).Execute(...)
	/Users/user/go/pkg/mod/github.com/spf13/[email protected]/command.go:864
github.com/gphotosuploader/gphotos-uploader-cli/internal/cmd.Execute()
	/Users/user/dev/vendor/gphotos-uploader-cli/internal/cmd/root.go:57 +0x25 fp=0xc00016df68 sp=0xc00016df28 pc=0x43bbba5
main.main()
	/Users/user/dev/vendor/gphotos-uploader-cli/main.go:10 +0x19 fp=0xc00016df80 sp=0xc00016df68 pc=0x43bc459
runtime.main()
	/Users/user/local/homebrew/Cellar/go/1.17/libexec/src/runtime/proc.go:255 +0x227 fp=0xc00016dfe0 sp=0xc00016df80 pc=0x4039747
runtime.goexit()
	/Users/user/local/homebrew/Cellar/go/1.17/libexec/src/runtime/asm_amd64.s:1581 +0x1 fp=0xc00016dfe8 sp=0xc00016dfe0 pc=0x4066fe1

goroutine 6 [select]:
github.com/syndtr/goleveldb/leveldb/util.(*BufferPool).drain(0xc00015c380)
	/Users/user/go/pkg/mod/github.com/syndtr/[email protected]/leveldb/util/buffer_pool.go:206 +0xcd
created by github.com/syndtr/goleveldb/leveldb/util.NewBufferPool
	/Users/user/go/pkg/mod/github.com/syndtr/[email protected]/leveldb/util/buffer_pool.go:237 +0x19b

goroutine 11 [select]:
github.com/syndtr/goleveldb/leveldb.(*DB).compactionError(0xc0001176c0)
	/Users/user/go/pkg/mod/github.com/syndtr/[email protected]/leveldb/db_compaction.go:90 +0x15e
created by github.com/syndtr/goleveldb/leveldb.openDB
	/Users/user/go/pkg/mod/github.com/syndtr/[email protected]/leveldb/db.go:142 +0x4ef

goroutine 12 [select]:
github.com/syndtr/goleveldb/leveldb.(*DB).mpoolDrain(0xc0001176c0)
	/Users/user/go/pkg/mod/github.com/syndtr/[email protected]/leveldb/db_state.go:101 +0xae
created by github.com/syndtr/goleveldb/leveldb.openDB
	/Users/user/go/pkg/mod/github.com/syndtr/[email protected]/leveldb/db.go:143 +0x531

goroutine 13 [select]:
github.com/syndtr/goleveldb/leveldb.(*DB).tCompaction(0xc0001176c0)
	/Users/user/go/pkg/mod/github.com/syndtr/[email protected]/leveldb/db_compaction.go:825 +0x685
created by github.com/syndtr/goleveldb/leveldb.openDB
	/Users/user/go/pkg/mod/github.com/syndtr/[email protected]/leveldb/db.go:149 +0x5a7

goroutine 14 [select]:
github.com/syndtr/goleveldb/leveldb.(*DB).mCompaction(0xc0001176c0)
	/Users/user/go/pkg/mod/github.com/syndtr/[email protected]/leveldb/db_compaction.go:762 +0x119
created by github.com/syndtr/goleveldb/leveldb.openDB
	/Users/user/go/pkg/mod/github.com/syndtr/[email protected]/leveldb/db.go:150 +0x5e9

goroutine 15 [select]:
github.com/syndtr/goleveldb/leveldb/util.(*BufferPool).drain(0xc00015c460)
	/Users/user/go/pkg/mod/github.com/syndtr/[email protected]/leveldb/util/buffer_pool.go:206 +0xcd
created by github.com/syndtr/goleveldb/leveldb/util.NewBufferPool
	/Users/user/go/pkg/mod/github.com/syndtr/[email protected]/leveldb/util/buffer_pool.go:237 +0x19b

goroutine 35 [select]:
github.com/syndtr/goleveldb/leveldb.(*DB).compactionError(0xc000603380)
	/Users/user/go/pkg/mod/github.com/syndtr/[email protected]/leveldb/db_compaction.go:90 +0x15e
created by github.com/syndtr/goleveldb/leveldb.openDB
	/Users/user/go/pkg/mod/github.com/syndtr/[email protected]/leveldb/db.go:142 +0x4ef

goroutine 36 [select]:
github.com/syndtr/goleveldb/leveldb.(*DB).mpoolDrain(0xc000603380)
	/Users/user/go/pkg/mod/github.com/syndtr/[email protected]/leveldb/db_state.go:101 +0xae
created by github.com/syndtr/goleveldb/leveldb.openDB
	/Users/user/go/pkg/mod/github.com/syndtr/[email protected]/leveldb/db.go:143 +0x531

goroutine 37 [select]:
github.com/syndtr/goleveldb/leveldb.(*DB).tCompaction(0xc000603380)
	/Users/user/go/pkg/mod/github.com/syndtr/[email protected]/leveldb/db_compaction.go:825 +0x685
created by github.com/syndtr/goleveldb/leveldb.openDB
	/Users/user/go/pkg/mod/github.com/syndtr/[email protected]/leveldb/db.go:149 +0x5a7

goroutine 38 [select]:
github.com/syndtr/goleveldb/leveldb.(*DB).mCompaction(0xc000603380)
	/Users/user/go/pkg/mod/github.com/syndtr/[email protected]/leveldb/db_compaction.go:762 +0x119
created by github.com/syndtr/goleveldb/leveldb.openDB
	/Users/user/go/pkg/mod/github.com/syndtr/[email protected]/leveldb/db.go:150 +0x5e9

goroutine 42 [IO wait]:
internal/poll.runtime_pollWait(0xd3a08a0, 0x72)
	/Users/user/local/homebrew/Cellar/go/1.17/libexec/src/runtime/netpoll.go:229 +0x89
internal/poll.(*pollDesc).wait(0xc000632100, 0xc0000d2000, 0x0)
	/Users/user/local/homebrew/Cellar/go/1.17/libexec/src/internal/poll/fd_poll_runtime.go:84 +0x32
internal/poll.(*pollDesc).waitRead(...)
	/Users/user/local/homebrew/Cellar/go/1.17/libexec/src/internal/poll/fd_poll_runtime.go:89
internal/poll.(*FD).Read(0xc000632100, {0xc0000d2000, 0xfd8, 0xfd8})
	/Users/user/local/homebrew/Cellar/go/1.17/libexec/src/internal/poll/fd_unix.go:167 +0x25a
net.(*netFD).Read(0xc000632100, {0xc0000d2000, 0xc0000d248c, 0x22})
	/Users/user/local/homebrew/Cellar/go/1.17/libexec/src/net/fd_posix.go:56 +0x29
net.(*conn).Read(0xc000620058, {0xc0000d2000, 0xc0010ec018, 0xc0011137f0})
	/Users/user/local/homebrew/Cellar/go/1.17/libexec/src/net/net.go:183 +0x45
crypto/tls.(*atLeastReader).Read(0xc00000e2d0, {0xc0000d2000, 0x0, 0x400daad})
	/Users/user/local/homebrew/Cellar/go/1.17/libexec/src/crypto/tls/conn.go:777 +0x3d
bytes.(*Buffer).ReadFrom(0xc0001105f8, {0x4518ec0, 0xc00000e2d0})
	/Users/user/local/homebrew/Cellar/go/1.17/libexec/src/bytes/buffer.go:204 +0x98
crypto/tls.(*Conn).readFromUntil(0xc000110380, {0xd32bd40, 0xc000620058}, 0xc0011138c8)
	/Users/user/local/homebrew/Cellar/go/1.17/libexec/src/crypto/tls/conn.go:799 +0xe5
crypto/tls.(*Conn).readRecordOrCCS(0xc000110380, 0x0)
	/Users/user/local/homebrew/Cellar/go/1.17/libexec/src/crypto/tls/conn.go:606 +0x112
crypto/tls.(*Conn).readRecord(...)
	/Users/user/local/homebrew/Cellar/go/1.17/libexec/src/crypto/tls/conn.go:574
crypto/tls.(*Conn).Read(0xc000110380, {0xc0010f0000, 0x1000, 0xc001113cd0})
	/Users/user/local/homebrew/Cellar/go/1.17/libexec/src/crypto/tls/conn.go:1277 +0x16f
bufio.(*Reader).Read(0xc0010a96e0, {0xc0010c2118, 0x9, 0xc001113cf8})
	/Users/user/local/homebrew/Cellar/go/1.17/libexec/src/bufio/bufio.go:227 +0x1b4
io.ReadAtLeast({0x4518d80, 0xc0010a96e0}, {0xc0010c2118, 0x9, 0x9}, 0x9)
	/Users/user/local/homebrew/Cellar/go/1.17/libexec/src/io/io.go:328 +0x9a
io.ReadFull(...)
	/Users/user/local/homebrew/Cellar/go/1.17/libexec/src/io/io.go:347
net/http.http2readFrameHeader({0xc0010c2118, 0x9, 0x42a15ab}, {0x4518d80, 0xc0010a96e0})
	/Users/user/local/homebrew/Cellar/go/1.17/libexec/src/net/http/h2_bundle.go:1553 +0x6e
net/http.(*http2Framer).ReadFrame(0xc0010c20e0)
	/Users/user/local/homebrew/Cellar/go/1.17/libexec/src/net/http/h2_bundle.go:1811 +0x95
net/http.(*http2clientConnReadLoop).run(0xc001113fa0)
	/Users/user/local/homebrew/Cellar/go/1.17/libexec/src/net/http/h2_bundle.go:8428 +0x165
net/http.(*http2ClientConn).readLoop(0xc0000c8180)
	/Users/user/local/homebrew/Cellar/go/1.17/libexec/src/net/http/h2_bundle.go:8350 +0x79
created by net/http.(*http2Transport).newClientConn
	/Users/user/local/homebrew/Cellar/go/1.17/libexec/src/net/http/h2_bundle.go:7302 +0xb45

[pacoorozco]

Hi @mlangenberg

Thanks a lot for all the information that you have already submitted. It has helped me to point where the problem is. Can you test something more?

1. Using the compiled package, configure it to use SecretsBackendType: file and run the gphotos-uploader-cli auth --debug to see if the password is asked and you are able to fill it.
2. Using your own compilation, do the same, SecretsBackendType: file and the auth to see the difference.

My guess is that homebrew is messing with stdin.

[mlangenberg]

Hey @pacoorozco, the 'file' secrets backend does work for the compiled homebrew package.

But when compiling from source myself, the 'file' secrets backend also returns the same error message and stracktrace as the 'keychain' secrets backend.

[mlangenberg]

FYI, after upgrading golang.org/x/term, the keychain secrets backend also works for my own compilation. Do you think that this will also fix the issue for the binary that is distributed through homebrew?

[pacoorozco]

Hi @mlangenberg

Yes, It could be... I was testing locally the "main" branch code and It was failing to me as-well. So there's a problem with the dependency. I'll do my best to update it an release a new version.

I'll let you know it to ask the favor of testing it...

Thanks a lot for your time investment.

[pacoorozco]

Hi @mlangenberg

I've merged your PR (thanks a lot) and I've released a new version. Would you be so kind to test the homebrew binary?*

[mlangenberg]

With v3.1.1 installed via homebrew, the keychain integration is still not working. Maybe it's a cross-compilation issue?

[error]  Token manager could not be started, err: Specified keyring backend not available
[fatal]  token manager could not be started, type:keychain, err: Specified keyring backend not available

See: 99designs/aws-vault#760

[pacoorozco]

Uhm... good finding @mlangenberg I'm not an expert in cross-compilation. As far as I seen this enables the use of CGO, which is required to compile go-keychain in a linux machine for a darwin architecture.

If I added it to my Makefile it raises an error, because obviously I don't have the specified SDK 🤔

I need to investigate more about this.

AFAIK compiling the main branch in your macOs should provide you a binary able to use keyring, Am I right?

[pacoorozco]

@mlangenberg Would you be able to test these binaries (wo/ homebrew).

[mlangenberg]

When I download the binaries, the keychain backend does not work.

gphotos-uploader-cli_3.2.0_darwin_amd64 ) ./gphotos-uploader-cli --config /tmp/gphotos-uploader-cli-2 --debug push
[info]   Reading configuration from '/tmp/gphotos-uploader-cli-2/config.hjson'
[debug]  Current configuration: {"APIAppCredentials":{"ClientID":"XXX","ClientSecret":"REMOVED"},"Account":"XXX","SecretsBackendType":"keychain","Jobs":[{"SourceFolder":"/Users/user/Pictures/gphotos-test","CreateAlbums":"folderName","DeleteAfterUpload":false,"IncludePatterns":[],"ExcludePatterns":[]}]}
[error]  Token manager could not be started, err: Specified keyring backend not available
[fatal]  token manager could not be started, type:keychain, err: Specified keyring backend not available

When I compile from source with make build:

./gphotos-uploader-cli --config /tmp/gphotos-uploader-cli-2 --debug push
[info]   Reading configuration from '/tmp/gphotos-uploader-cli-2/config.hjson'
[debug]  Current configuration: {"APIAppCredentials":{"ClientID":"XXX","ClientSecret":"REMOVED"},"Account":"XXX","SecretsBackendType":"keychain","Jobs":[{"SourceFolder":"/Users/user/Pictures/gphotos-test","CreateAlbums":"folderName","DeleteAfterUpload":false,"IncludePatterns":[],"ExcludePatterns":[]}]}
[info]   Getting OAuth token for 'XXX'
[done] √ Token is valid, expires at 2021-09-13 20:47:28.841244 +0200 CEST

I'm sorry that we are not making much progress on this. To be honest, my workaround to set GPHOTOS_CLI_TOKENSTORE_KEY directly from the keychain before starting the program is fine for me. So it's not a big deal if it isn't supported directly.

I did come across this comment: 99designs/keyring#78 (comment)