tsh login fails with "failed to get assertion: internal error" with older U2F devices

[ravicious]

Expected behavior:

tsh login works with an older U2F device (FIDO U2F Security Key, Firmware 4.1.8). It's the one that GitHub was giving away at a discount in 2015.

Current behavior:

tsh login returns an error, while the key works just fine in the browser.

$ tsh login --proxy <proxy> -d
2024-08-01T08:57:25+02:00 DEBU [KEYSTORE]  Reading certificates from path "/Users/rav/.tsh/keys/<proxy>/rav-ssh/<proxy>-cert.pub". client/keystore.go:357
2024-08-01T08:57:25+02:00 DEBU [KEYSTORE]  Teleport TLS certificate valid until "2024-07-17 20:44:36 +0000 UTC". client/client_store.go:118
2024-08-01T08:57:25+02:00 INFO [CLIENT]    ALPN connection upgrade required for "<proxy>:443": true. client/api.go:819
2024-08-01T08:57:25+02:00 INFO [CLIENT]    no host login given. defaulting to rav client/api.go:1162
2024-08-01T08:57:25+02:00 INFO [CLIENT]    [KEY AGENT] Connected to the system agent: "/private/tmp/com.apple.launchd.cBB1s29dvD/Listeners" client/api.go:4553
2024-08-01T08:57:25+02:00 DEBU [KEYSTORE]  Reading certificates from path "/Users/rav/.tsh/keys/<proxy>/rav-ssh/<proxy>-cert.pub". client/keystore.go:357
2024-08-01T08:57:25+02:00 DEBU [KEYSTORE]  Teleport TLS certificate valid until "2024-07-17 20:44:36 +0000 UTC". client/client_store.go:118
2024-08-01T08:57:25+02:00 INFO [KEYAGENT]  Loading SSH key for user "rav" and cluster "<proxy>". client/keyagent.go:198
2024-08-01T08:57:25+02:00 DEBU [CLIENT]    not using loopback pool for remote proxy addr: <proxy>:443 client/api.go:4508
2024-08-01T08:57:25+02:00 DEBU  Attempting request to Proxy web api method:GET host:<proxy>:443 path:/webapi/ping trace_id:8dffac01733688a94e09d2ba4eafe779 span_id:08ec9e78577f9c11 webclient/webclient.go:131
2024-08-01T08:57:25+02:00 DEBU  ALPN connection upgrade test complete address:<proxy>:443 upgrade_required:true trace_id:8dffac01733688a94e09d2ba4eafe779 span_id:08ec9e78577f9c11 client/alpn_conn_upgrade.go:96
2024-08-01T08:57:25+02:00 DEBU [CLIENT]    Attempting to login with a new RSA private key. client/api.go:3778
Enter password for Teleport user rav:
2024-08-01T08:57:28+02:00 DEBU [CLIENT]    not using loopback pool for remote proxy addr: <proxy>:443 client/api.go:4508
2024-08-01T08:57:28+02:00 DEBU [CLIENT]    HTTPS client init(proxyAddr=<proxy>:443, insecure=false, extraHeaders=map[]) client/weblogin.go:354
2024-08-01T08:57:28+02:00 DEBU             Attempting platform login webauthncli/api.go:168
2024-08-01T08:57:28+02:00 DEBU             Platform login failed, falling back to cross-platform error:[touch ID not available] webauthncli/api.go:174
2024-08-01T08:57:28+02:00 DEBU             FIDO2: Using libfido2 for assertion webauthncli/api.go:183
2024-08-01T08:57:28+02:00 DEBU             FIDO2: assertion: passwordless=false, uv=false, 3 allowed credentials webauthncli/fido2.go:167
Tap any security key
2024-08-01T08:57:28+02:00 DEBU             FIDO2: Device ioreg://4295061609: not a FIDO2 device webauthncli/fido2.go:804
2024-08-01T08:57:30+02:00 DEBU             FIDO2: Device ioreg://4295061609: callback returned, requiresPIN=false, err=failed to get assertion: internal error webauthncli/fido2.go:825
2024-08-01T08:57:30+02:00 DEBU             FIDO2: Close device ioreg://4295061609, err=<nil> webauthncli/fido2.go:784
2024-08-01T08:57:30+02:00 DEBU             FIDO2: Cancel device ioreg://4295061609, err=<nil> webauthncli/fido2.go:768
2024-08-01T08:57:30+02:00 DEBU             FIDO2: Device goroutines exited cleanly webauthncli/fido2.go:630

ERROR REPORT:
Original Error: trace.aggregate Webauthn authentication failed
	failed to get assertion: internal error
Stack Trace:
	github.com/gravitational/teleport/lib/client/mfa/prompt.go:164 github.com/gravitational/teleport/lib/client/mfa.HandleMFAPromptGoroutines
	github.com/gravitational/teleport/lib/client/mfa/cli.go:110 github.com/gravitational/teleport/lib/client/mfa.(*CLIPrompt).Run
	github.com/gravitational/teleport/lib/client/weblogin.go:664 github.com/gravitational/teleport/lib/client.SSHAgentMFALogin
	github.com/gravitational/teleport/lib/client/api.go:3918 github.com/gravitational/teleport/lib/client.(*TeleportClient).mfaLocalLogin
	github.com/gravitational/teleport/lib/client/api.go:3847 github.com/gravitational/teleport/lib/client.(*TeleportClient).localLogin
	github.com/gravitational/teleport/lib/client/api.go:3475 github.com/gravitational/teleport/lib/client.(*TeleportClient).getSSHLoginFunc.func2
	github.com/gravitational/teleport/lib/client/api.go:3676 github.com/gravitational/teleport/lib/client.(*TeleportClient).SSHLogin.func1
	github.com/gravitational/teleport/lib/client/api.go:3723 github.com/gravitational/teleport/lib/client.(*TeleportClient).loginWithHardwareKeyRetry
	github.com/gravitational/teleport/lib/client/api.go:3674 github.com/gravitational/teleport/lib/client.(*TeleportClient).SSHLogin
	github.com/gravitational/teleport/lib/client/api.go:3255 github.com/gravitational/teleport/lib/client.(*TeleportClient).Login
	github.com/gravitational/teleport/tool/tsh/common/tsh.go:1942 github.com/gravitational/teleport/tool/tsh/common.onLogin
	github.com/gravitational/teleport/tool/tsh/common/tsh.go:1433 github.com/gravitational/teleport/tool/tsh/common.Run
	github.com/gravitational/teleport/tool/tsh/common/tsh.go:608 github.com/gravitational/teleport/tool/tsh/common.Main
	github.com/gravitational/teleport/tool/tsh/main.go:26 main.main
	runtime/proc.go:271 runtime.main
	runtime/asm_amd64.s:1695 runtime.goexit
User Message: failed to authenticate using available MFA devices
	Webauthn authentication failed
	failed to get assertion: internal error

Bug details:

• Teleport version: v16.1.1

[codingllama]

Hey Rafal, I tried a repro with my oldest key (an U2F yubi4, firmware 4.3.7), but it works fine for me.

Is the failure consistent?

[codingllama]

(send comment too soon)

Is the failure consistent?

Can you repro with older Teleport versions or did it work before?

[ravicious]

The failure is consistent. I'm quite certain that it did work – it's one of the few keys that I own that fits into my old Intel MBP.

I just downloaded v15.0.0, v14.0.0, and v13.0.0 and they all don't even make the key blink during tsh login.

If I switch to the v15.0.0 tag and build tsh with FIDO2=no, the key works with tsh. If I build tsh on this tag with FIDO2=dynamic, it doesn't blink. I guess it might have something to do with the U2F fallback that we've removed?

[codingllama]

Update, in case anyone else is following this: Rafal has a repro, we've been talking about this the last few days. It seems like his yubi4 errors when certain allowed credentials are present. We are looking into some possible ideas for a fix.

[codingllama]

I'm keeping this open for an eventual backport after the v17 testplan.

[ravicious]

I think we can close this now, right?