-
Notifications
You must be signed in to change notification settings - Fork 3.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
jwtauthccl : http client missing support for proxies #123575
Comments
This has happened a couple of times, and there are other inconsistencies in the HTTP clients we use (notably, the OIDC client library uses the default http transports which support proxies but don't have the lower timeouts we use elsewhere). We should try to centralize our use of http clients in 1 (or a few) places. This is tricky because some libraries access the default client directly. We need to either modify the global defaults to have the configuration we want (such as timeouts) or do something to disallow the use of global defaults (installing some sort of poisoned objects that will crash when used?) |
Is there something we can do on lint / static check side? at least for direct HTTP client usage in our code. |
We are running into issues with jwt authentication and currently unable to provide support as we are not logging the error from the http client used in the authenticator. Thus, the PR looks to propagate the obtained error from `ValidateJWTLogin`. We are also introducing `JWTClientTimeout` and `JWTClientCustomCA` cluster settings such that these could be configured directly for the http client used in authenticator. The http client now also respects the system http proxy if set. fixes cockroachdb#123575, Epic CRDB-38386, Release note(security update): We are adding 2 cluster settings `server.jwt_authentication.client.timeout` and `server.jwt_authentication.client.custom_ca` which can control the jwt auth behaviour for http client calls.
We are running into issues with jwt authentication and currently unable to provide support as we are not logging the error from the http client used in the authenticator. The PR looks to propagate this obtained error from `ValidateJWTLogin` http client. We are also introducing `JWTClientTimeout` and `JWTClientCustomCA` cluster settings such that these could be configured directly for the http client used in authenticator. The http client now also respects the system http proxy if set and has a cluster setting `JWTClientSystemProxyEnabled` to enable it. Validated the error details when presenting an expired token ``` ERROR: JWT authentication: invalid token SQLSTATE: 28000 DETAIL: unable to parse token: exp not satisfied Failed running "sql" ``` Validated error on setting wrong proxy params ``` ERROR: JWT authentication: unable to validate token SQLSTATE: 28000 DETAIL: unable to fetch jwks: Get "https://accounts.google.com/.well-known/openid-configuration": proxyconnect tcp: dial tcp [::1]:3129: connect: connection refused Failed running "sql" ``` Verified access logs after setting up squid proxy and passing env HTTP_PROXY and HTTPS_PROXY params ``` 1715103871.761 144 ::1 TCP_TUNNEL/200 5708 CONNECT accounts.google.com:443 - HIER_DIRECT/74.125.200.84 - 1715103871.836 73 ::1 TCP_TUNNEL/200 5964 CONNECT www.googleapis.com:443 - HIER_DIRECT/142.250.182.10 - ``` fixes cockroachdb#123575, Epic CRDB-38386, Release note(security update): We are adding 3 cluster settings `server.jwt_authentication.client.timeout`, `server.jwt_authentication.client.custom_ca` and `server.jwt_authentication.client.system_proxy.enabled` which can control the jwt auth behaviour for http client calls.
We are running into issues with jwt authentication and currently unable to provide support as we are not logging the error from the http client used in the authenticator. The PR looks to propagate this obtained error from `ValidateJWTLogin` http client. We are also introducing `JWTClientTimeout` and `JWTClientCustomCA` cluster settings such that these could be configured directly for the http client used in authenticator. The http client now also respects the system http proxy if set and has a cluster setting `JWTClientSystemProxyEnabled` to enable it. Validated the error details when presenting an expired token ``` ERROR: JWT authentication: invalid token SQLSTATE: 28000 DETAIL: unable to parse token: exp not satisfied Failed running "sql" ``` Validated error on setting wrong proxy params ``` ERROR: JWT authentication: unable to validate token SQLSTATE: 28000 DETAIL: unable to fetch jwks: Get "https://accounts.google.com/.well-known/openid-configuration": proxyconnect tcp: dial tcp [::1]:3129: connect: connection refused Failed running "sql" ``` Verified access logs after setting up squid proxy and passing env HTTP_PROXY and HTTPS_PROXY params ``` 1715103871.761 144 ::1 TCP_TUNNEL/200 5708 CONNECT accounts.google.com:443 - HIER_DIRECT/74.125.200.84 - 1715103871.836 73 ::1 TCP_TUNNEL/200 5964 CONNECT www.googleapis.com:443 - HIER_DIRECT/142.250.182.10 - ``` fixes cockroachdb#123575, Epic CRDB-38386, Release note(security update): We are adding 3 cluster settings `server.jwt_authentication.client.timeout`, `server.jwt_authentication.client.custom_ca` and `server.jwt_authentication.client.system_proxy.enabled` which can control the jwt auth behaviour for http client calls.
We are running into issues with jwt authentication and currently unable to provide support as we are not logging the error from the http client used in the authenticator. The PR looks to propagate this obtained error from `ValidateJWTLogin` http client. We are also introducing `JWTClientCustomCA` cluster setting such that this could be configured directly for the http client used in authenticator. The http client now also respects the system http proxy if set. Validated the error details when presenting an expired token ``` ERROR: JWT authentication: invalid token SQLSTATE: 28000 DETAIL: unable to parse token: exp not satisfied Failed running "sql" ``` Validated error on setting wrong proxy params ``` ERROR: JWT authentication: unable to validate token SQLSTATE: 28000 DETAIL: unable to fetch jwks: Get "https://accounts.google.com/.well-known/openid-configuration": proxyconnect tcp: dial tcp [::1]:3129: connect: connection refused Failed running "sql" ``` Verified access logs after setting up squid proxy and passing env HTTP_PROXY and HTTPS_PROXY params ``` 1715103871.761 144 ::1 TCP_TUNNEL/200 5708 CONNECT accounts.google.com:443 - HIER_DIRECT/74.125.200.84 - 1715103871.836 73 ::1 TCP_TUNNEL/200 5964 CONNECT www.googleapis.com:443 - HIER_DIRECT/142.250.182.10 - ``` fixes cockroachdb#123575, Epic CRDB-38386, Release note(security update): We are adding a cluster settings `server.jwt_authentication.client.custom_ca` which can tune the jwt auth behaviour when we need custom ca http client calls.
We are running into issues with jwt authentication and currently unable to provide support as we are not logging the error from the http client used in the authenticator. The PR looks to propagate this obtained error from `ValidateJWTLogin` http client. We are also introducing `JWTClientCustomCA` cluster setting such that this could be configured directly for the http client used in authenticator. The http client now also respects the system http proxy if set. Validated the error details when presenting an expired token ``` ERROR: JWT authentication: invalid token SQLSTATE: 28000 DETAIL: unable to parse token: exp not satisfied Failed running "sql" ``` Validated error on setting wrong proxy params ``` ERROR: JWT authentication: unable to validate token SQLSTATE: 28000 DETAIL: unable to fetch jwks: Get "https://accounts.google.com/.well-known/openid-configuration": proxyconnect tcp: dial tcp [::1]:3129: connect: connection refused Failed running "sql" ``` Verified access logs after setting up squid proxy and passing env HTTP_PROXY and HTTPS_PROXY params ``` 1715103871.761 144 ::1 TCP_TUNNEL/200 5708 CONNECT accounts.google.com:443 - HIER_DIRECT/74.125.200.84 - 1715103871.836 73 ::1 TCP_TUNNEL/200 5964 CONNECT www.googleapis.com:443 - HIER_DIRECT/142.250.182.10 - ``` fixes cockroachdb#123575, Epic CRDB-38386, Release note(security update): We are adding a cluster settings `server.jwt_authentication.client.custom_ca` which can tune the jwt auth behaviour when we need custom ca http client calls.
We are running into issues with jwt authentication and currently unable to provide support as we are not logging the error from the http client used in the authenticator. The PR looks to propagate this obtained error from `ValidateJWTLogin` http client. We are also introducing `JWTClientCustomCA` cluster setting such that this could be configured directly for the http client used in authenticator. The http client now also respects the system http proxy if set. Validated the error details when presenting an expired token ``` ERROR: JWT authentication: invalid token SQLSTATE: 28000 DETAIL: unable to parse token: exp not satisfied Failed running "sql" ``` Validated error on setting wrong proxy params ``` ERROR: JWT authentication: unable to validate token SQLSTATE: 28000 Failed running "sql" ``` and logged error: ``` I240510 08:31:28.604141 1473 4@util/log/event_log.go:32 ⋮ [T1,Vsystem,n1,client=127.0.0.1:56289,hostssl,user=‹sourav.sarangi›] 3 ={"Timestamp":1715329888604122000,"EventType":"client_authentication_failed","InstanceID":1,"Network":"tcp","RemoteAddress":"‹127.0.0.1:56289›","SessionID":"17ce136f2a8ecd480000000000000001","Transport":"hostssl","User":"‹sourav.sarangi›","SystemIdentity":"‹sourav.sarangi›","Reason":"CREDENTIALS_INVALID","Detail":"JWT authentication: unable to validate token\nunable to fetch jwks: Get \"https://accounts.google.com/.well-known/openid-configuration\": proxyconnect tcp: dial tcp [::1]:3129: connect: connection refused","Method":"jwt_token"} ``` Verified access logs after setting up squid proxy and passing env HTTP_PROXY and HTTPS_PROXY params ``` 1715103871.761 144 ::1 TCP_TUNNEL/200 5708 CONNECT accounts.google.com:443 - HIER_DIRECT/74.125.200.84 - 1715103871.836 73 ::1 TCP_TUNNEL/200 5964 CONNECT www.googleapis.com:443 - HIER_DIRECT/142.250.182.10 - ``` fixes cockroachdb#123575, Epic CRDB-38386, Release note(security update): We are adding a cluster settings `server.jwt_authentication.client.custom_ca` which can tune the jwt auth behaviour when we need custom ca http client calls.
We are running into issues with jwt authentication and currently unable to provide support as we are not logging the error from the http client used in the authenticator. The PR looks to propagate this obtained error from `ValidateJWTLogin` http client. We are also introducing `JWTClientCustomCA` cluster setting such that this could be configured directly for the http client used in authenticator. The http client now also respects the system http proxy if set. Validated the error details when presenting an expired token ``` ERROR: JWT authentication: invalid token SQLSTATE: 28000 DETAIL: unable to parse token: exp not satisfied Failed running "sql" ``` Validated error on setting wrong proxy params ``` ERROR: JWT authentication: unable to validate token SQLSTATE: 28000 Failed running "sql" ``` and logged error: ``` I240510 08:31:28.604141 1473 4@util/log/event_log.go:32 ⋮ [T1,Vsystem,n1,client=127.0.0.1:56289,hostssl,user=‹sourav.sarangi›] 3 ={"Timestamp":1715329888604122000,"EventType":"client_authentication_failed","InstanceID":1,"Network":"tcp","RemoteAddress":"‹127.0.0.1:56289›","SessionID":"17ce136f2a8ecd480000000000000001","Transport":"hostssl","User":"‹sourav.sarangi›","SystemIdentity":"‹sourav.sarangi›","Reason":"CREDENTIALS_INVALID","Detail":"JWT authentication: unable to validate token\nunable to fetch jwks: Get \"https://accounts.google.com/.well-known/openid-configuration\": proxyconnect tcp: dial tcp [::1]:3129: connect: connection refused","Method":"jwt_token"} ``` Verified access logs after setting up squid proxy and passing env HTTP_PROXY and HTTPS_PROXY params ``` 1715103871.761 144 ::1 TCP_TUNNEL/200 5708 CONNECT accounts.google.com:443 - HIER_DIRECT/74.125.200.84 - 1715103871.836 73 ::1 TCP_TUNNEL/200 5964 CONNECT www.googleapis.com:443 - HIER_DIRECT/142.250.182.10 - ``` fixes cockroachdb#123575, Epic CRDB-38386, CRDB-38408 Release note(security update): We are adding a cluster settings `server.jwt_authentication.client.custom_ca` which can tune the jwt auth behaviour when we need custom ca http client calls.
We are running into issues with jwt authentication and currently unable to provide support as we are not logging the error from the http client used in the authenticator. The PR looks to propagate this obtained error from `ValidateJWTLogin` http client. We are also introducing `JWTClientCustomCA` cluster setting such that this could be configured directly for the http client used in authenticator. The http client now also respects the system http proxy if set. Validated the error details when presenting an expired token ``` ERROR: JWT authentication: invalid token SQLSTATE: 28000 DETAIL: unable to parse token: exp not satisfied Failed running "sql" ``` Validated error on setting wrong proxy params ``` ERROR: JWT authentication: unable to validate token SQLSTATE: 28000 Failed running "sql" ``` and logged error: ``` I240510 08:31:28.604141 1473 4@util/log/event_log.go:32 ⋮ [T1,Vsystem,n1,client=127.0.0.1:56289,hostssl,user=‹sourav.sarangi›] 3 ={"Timestamp":1715329888604122000,"EventType":"client_authentication_failed","InstanceID":1,"Network":"tcp","RemoteAddress":"‹127.0.0.1:56289›","SessionID":"17ce136f2a8ecd480000000000000001","Transport":"hostssl","User":"‹sourav.sarangi›","SystemIdentity":"‹sourav.sarangi›","Reason":"CREDENTIALS_INVALID","Detail":"JWT authentication: unable to validate token\nunable to fetch jwks: Get \"https://accounts.google.com/.well-known/openid-configuration\": proxyconnect tcp: dial tcp [::1]:3129: connect: connection refused","Method":"jwt_token"} ``` Verified access logs after setting up squid proxy and passing env HTTP_PROXY and HTTPS_PROXY params ``` 1715103871.761 144 ::1 TCP_TUNNEL/200 5708 CONNECT accounts.google.com:443 - HIER_DIRECT/74.125.200.84 - 1715103871.836 73 ::1 TCP_TUNNEL/200 5964 CONNECT www.googleapis.com:443 - HIER_DIRECT/142.250.182.10 - ``` fixes cockroachdb#123575, CRDB-38386, CRDB-38408 Epic None Release note(security update): We are adding a cluster settings `server.jwt_authentication.client.custom_ca` which can tune the jwt auth behaviour when we need custom ca http client calls.
We are running into issues with jwt authentication and currently unable to provide support as we are not logging the error from the http client used in the authenticator. The PR looks to propagate this obtained error from `ValidateJWTLogin` http client. The http client now also respects the system http proxy if set. Validated the error details when presenting an expired token ``` ERROR: JWT authentication: invalid token SQLSTATE: 28000 DETAIL: unable to parse token: exp not satisfied Failed running "sql" ``` Validated error on setting wrong proxy params ``` ERROR: JWT authentication: unable to validate token SQLSTATE: 28000 Failed running "sql" ``` and logged error: ``` I240510 08:31:28.604141 1473 4@util/log/event_log.go:32 ⋮ [T1,Vsystem,n1,client=127.0.0.1:56289,hostssl,user=‹sourav.sarangi›] 3 ={"Timestamp":1715329888604122000,"EventType":"client_authentication_failed","InstanceID":1,"Network":"tcp","RemoteAddress":"‹127.0.0.1:56289›","SessionID":"17ce136f2a8ecd480000000000000001","Transport":"hostssl","User":"‹sourav.sarangi›","SystemIdentity":"‹sourav.sarangi›","Reason":"CREDENTIALS_INVALID","Detail":"JWT authentication: unable to validate token\nunable to fetch jwks: Get \"https://accounts.google.com/.well-known/openid-configuration\": proxyconnect tcp: dial tcp [::1]:3129: connect: connection refused","Method":"jwt_token"} ``` Verified access logs after setting up squid proxy and passing env HTTP_PROXY and HTTPS_PROXY params ``` 1715103871.761 144 ::1 TCP_TUNNEL/200 5708 CONNECT accounts.google.com:443 - HIER_DIRECT/74.125.200.84 - 1715103871.836 73 ::1 TCP_TUNNEL/200 5964 CONNECT www.googleapis.com:443 - HIER_DIRECT/142.250.182.10 - ``` fixes cockroachdb#123575, CRDB-38386, CRDB-38408 Epic None Release note: None
123697: ccl,sql,util: Fix jwt auth and add sensitive error logs r=souravcrl a=souravcrl ccl,sql,util: Fix jwt auth and add sensitive error logs We are running into issues with jwt authentication and currently unable to provide support as we are not logging the error from the http client used in the authenticator. The PR looks to propagate this obtained error from `ValidateJWTLogin` http client. The http client now also respects the system http proxy if set. Validated the error details when presenting an expired token ``` ERROR: JWT authentication: invalid token SQLSTATE: 28000 DETAIL: unable to parse token: exp not satisfied Failed running "sql" ``` Validated error on setting wrong proxy params ``` ERROR: JWT authentication: unable to validate token SQLSTATE: 28000 Failed running "sql" ``` and logged error: ``` I240510 08:31:28.604141 1473 4@util/log/event_log.go:32 ⋮ [T1,Vsystem,n1,client=127.0.0.1:56289,hostssl,user=‹sourav.sarangi›] 3 ={"Timestamp":1715329888604122000,"EventType":"client_authentication_failed","InstanceID":1,"Network":"tcp","RemoteAddress":"‹127.0.0.1:56289›","SessionID":"17ce136f2a8ecd480000000000000001","Transport":"hostssl","User":"‹sourav.sarangi›","SystemIdentity":"‹sourav.sarangi›","Reason":"CREDENTIALS_INVALID","Detail":"JWT authentication: unable to validate token\nunable to fetch jwks: Get \"https://accounts.google.com/.well-known/openid-configuration\": proxyconnect tcp: dial tcp [::1]:3129: connect: connection refused","Method":"jwt_token"} ``` Verified access logs after setting up squid proxy and passing env HTTP_PROXY and HTTPS_PROXY params ``` 1715103871.761 144 ::1 TCP_TUNNEL/200 5708 CONNECT accounts.google.com:443 - HIER_DIRECT/74.125.200.84 - 1715103871.836 73 ::1 TCP_TUNNEL/200 5964 CONNECT www.googleapis.com:443 - HIER_DIRECT/142.250.182.10 - ``` fixes #123575, CRDB-38386, CRDB-38408 Epic None Release note: Noneccl,sql,util: Fix jwt auth and add sensitive error logs We are running into issues with jwt authentication and currently unable to provide support as we are not logging the error from the http client used in the authenticator. The PR looks to propagate this obtained error from `ValidateJWTLogin` http client. The http client now also respects the system http proxy if set. Validated the error details when presenting an expired token ``` ERROR: JWT authentication: invalid token SQLSTATE: 28000 DETAIL: unable to parse token: exp not satisfied Failed running "sql" ``` Validated error on setting wrong proxy params ``` ERROR: JWT authentication: unable to validate token SQLSTATE: 28000 Failed running "sql" ``` and logged error: ``` I240510 08:31:28.604141 1473 4@util/log/event_log.go:32 ⋮ [T1,Vsystem,n1,client=127.0.0.1:56289,hostssl,user=‹sourav.sarangi›] 3 ={"Timestamp":1715329888604122000,"EventType":"client_authentication_failed","InstanceID":1,"Network":"tcp","RemoteAddress":"‹127.0.0.1:56289›","SessionID":"17ce136f2a8ecd480000000000000001","Transport":"hostssl","User":"‹sourav.sarangi›","SystemIdentity":"‹sourav.sarangi›","Reason":"CREDENTIALS_INVALID","Detail":"JWT authentication: unable to validate token\nunable to fetch jwks: Get \"https://accounts.google.com/.well-known/openid-configuration\": proxyconnect tcp: dial tcp [::1]:3129: connect: connection refused","Method":"jwt_token"} ``` Verified access logs after setting up squid proxy and passing env HTTP_PROXY and HTTPS_PROXY params ``` 1715103871.761 144 ::1 TCP_TUNNEL/200 5708 CONNECT accounts.google.com:443 - HIER_DIRECT/74.125.200.84 - 1715103871.836 73 ::1 TCP_TUNNEL/200 5964 CONNECT www.googleapis.com:443 - HIER_DIRECT/142.250.182.10 - ``` fixes #123575, CRDB-38386, CRDB-38408 Epic None Release note: None Co-authored-by: Sourav Sarangi <[email protected]>
We are running into issues with jwt authentication and currently unable to provide support as we are not logging the error from the http client used in the authenticator. The PR looks to propagate this obtained error from `ValidateJWTLogin` http client. The http client now also respects the system http proxy if set. Validated the error details when presenting an expired token ``` ERROR: JWT authentication: invalid token SQLSTATE: 28000 DETAIL: unable to parse token: exp not satisfied Failed running "sql" ``` Validated error on setting wrong proxy params ``` ERROR: JWT authentication: unable to validate token SQLSTATE: 28000 Failed running "sql" ``` and logged error: ``` I240510 08:31:28.604141 1473 4@util/log/event_log.go:32 ⋮ [T1,Vsystem,n1,client=127.0.0.1:56289,hostssl,user=‹sourav.sarangi›] 3 ={"Timestamp":1715329888604122000,"EventType":"client_authentication_failed","InstanceID":1,"Network":"tcp","RemoteAddress":"‹127.0.0.1:56289›","SessionID":"17ce136f2a8ecd480000000000000001","Transport":"hostssl","User":"‹sourav.sarangi›","SystemIdentity":"‹sourav.sarangi›","Reason":"CREDENTIALS_INVALID","Detail":"JWT authentication: unable to validate token\nunable to fetch jwks: Get \"https://accounts.google.com/.well-known/openid-configuration\": proxyconnect tcp: dial tcp [::1]:3129: connect: connection refused","Method":"jwt_token"} ``` Verified access logs after setting up squid proxy and passing env HTTP_PROXY and HTTPS_PROXY params ``` 1715103871.761 144 ::1 TCP_TUNNEL/200 5708 CONNECT accounts.google.com:443 - HIER_DIRECT/74.125.200.84 - 1715103871.836 73 ::1 TCP_TUNNEL/200 5964 CONNECT www.googleapis.com:443 - HIER_DIRECT/142.250.182.10 - ``` fixes cockroachdb#123575, CRDB-38386, CRDB-38408 Epic None Release note: None
We are running into issues with jwt authentication and currently unable to provide support as we are not logging the error from the http client used in the authenticator. The PR looks to propagate this obtained error from `ValidateJWTLogin` http client. The http client now also respects the system http proxy if set. Validated the error details when presenting an expired token ``` ERROR: JWT authentication: invalid token SQLSTATE: 28000 DETAIL: unable to parse token: exp not satisfied Failed running "sql" ``` Validated error on setting wrong proxy params ``` ERROR: JWT authentication: unable to validate token SQLSTATE: 28000 Failed running "sql" ``` and logged error: ``` I240510 08:31:28.604141 1473 4@util/log/event_log.go:32 ⋮ [T1,Vsystem,n1,client=127.0.0.1:56289,hostssl,user=‹sourav.sarangi›] 3 ={"Timestamp":1715329888604122000,"EventType":"client_authentication_failed","InstanceID":1,"Network":"tcp","RemoteAddress":"‹127.0.0.1:56289›","SessionID":"17ce136f2a8ecd480000000000000001","Transport":"hostssl","User":"‹sourav.sarangi›","SystemIdentity":"‹sourav.sarangi›","Reason":"CREDENTIALS_INVALID","Detail":"JWT authentication: unable to validate token\nunable to fetch jwks: Get \"https://accounts.google.com/.well-known/openid-configuration\": proxyconnect tcp: dial tcp [::1]:3129: connect: connection refused","Method":"jwt_token"} ``` Verified access logs after setting up squid proxy and passing env HTTP_PROXY and HTTPS_PROXY params ``` 1715103871.761 144 ::1 TCP_TUNNEL/200 5708 CONNECT accounts.google.com:443 - HIER_DIRECT/74.125.200.84 - 1715103871.836 73 ::1 TCP_TUNNEL/200 5964 CONNECT www.googleapis.com:443 - HIER_DIRECT/142.250.182.10 - ``` fixes cockroachdb#123575, CRDB-38386, CRDB-38408 Epic None Release note: None
We are running into issues with jwt authentication and currently unable to provide support as we are not logging the error from the http client used in the authenticator. The PR looks to propagate this obtained error from `ValidateJWTLogin` http client. The http client now also respects the system http proxy if set. Validated the error details when presenting an expired token ``` ERROR: JWT authentication: invalid token SQLSTATE: 28000 DETAIL: unable to parse token: exp not satisfied Failed running "sql" ``` Validated error on setting wrong proxy params ``` ERROR: JWT authentication: unable to validate token SQLSTATE: 28000 Failed running "sql" ``` and logged error: ``` I240510 08:31:28.604141 1473 4@util/log/event_log.go:32 ⋮ [T1,Vsystem,n1,client=127.0.0.1:56289,hostssl,user=‹sourav.sarangi›] 3 ={"Timestamp":1715329888604122000,"EventType":"client_authentication_failed","InstanceID":1,"Network":"tcp","RemoteAddress":"‹127.0.0.1:56289›","SessionID":"17ce136f2a8ecd480000000000000001","Transport":"hostssl","User":"‹sourav.sarangi›","SystemIdentity":"‹sourav.sarangi›","Reason":"CREDENTIALS_INVALID","Detail":"JWT authentication: unable to validate token\nunable to fetch jwks: Get \"https://accounts.google.com/.well-known/openid-configuration\": proxyconnect tcp: dial tcp [::1]:3129: connect: connection refused","Method":"jwt_token"} ``` Verified access logs after setting up squid proxy and passing env HTTP_PROXY and HTTPS_PROXY params ``` 1715103871.761 144 ::1 TCP_TUNNEL/200 5708 CONNECT accounts.google.com:443 - HIER_DIRECT/74.125.200.84 - 1715103871.836 73 ::1 TCP_TUNNEL/200 5964 CONNECT www.googleapis.com:443 - HIER_DIRECT/142.250.182.10 - ``` fixes #123575, CRDB-38386, CRDB-38408 Epic None Release note: None
We are running into issues with jwt authentication and currently unable to provide support as we are not logging the error from the http client used in the authenticator. The PR looks to propagate this obtained error from `ValidateJWTLogin` http client. The http client now also respects the system http proxy if set. Validated the error details when presenting an expired token ``` ERROR: JWT authentication: invalid token SQLSTATE: 28000 DETAIL: unable to parse token: exp not satisfied Failed running "sql" ``` Validated error on setting wrong proxy params ``` ERROR: JWT authentication: unable to validate token SQLSTATE: 28000 Failed running "sql" ``` and logged error: ``` I240510 08:31:28.604141 1473 4@util/log/event_log.go:32 ⋮ [T1,Vsystem,n1,client=127.0.0.1:56289,hostssl,user=‹sourav.sarangi›] 3 ={"Timestamp":1715329888604122000,"EventType":"client_authentication_failed","InstanceID":1,"Network":"tcp","RemoteAddress":"‹127.0.0.1:56289›","SessionID":"17ce136f2a8ecd480000000000000001","Transport":"hostssl","User":"‹sourav.sarangi›","SystemIdentity":"‹sourav.sarangi›","Reason":"CREDENTIALS_INVALID","Detail":"JWT authentication: unable to validate token\nunable to fetch jwks: Get \"https://accounts.google.com/.well-known/openid-configuration\": proxyconnect tcp: dial tcp [::1]:3129: connect: connection refused","Method":"jwt_token"} ``` Verified access logs after setting up squid proxy and passing env HTTP_PROXY and HTTPS_PROXY params ``` 1715103871.761 144 ::1 TCP_TUNNEL/200 5708 CONNECT accounts.google.com:443 - HIER_DIRECT/74.125.200.84 - 1715103871.836 73 ::1 TCP_TUNNEL/200 5964 CONNECT www.googleapis.com:443 - HIER_DIRECT/142.250.182.10 - ``` fixes cockroachdb#123575, CRDB-38386, CRDB-38408 Epic None Release note: None
We are running into issues with jwt authentication and currently unable to provide support as we are not logging the error from the http client used in the authenticator. The PR looks to propagate this obtained error from `ValidateJWTLogin` http client. The http client now also respects the system http proxy if set. Validated the error details when presenting an expired token ``` ERROR: JWT authentication: invalid token SQLSTATE: 28000 DETAIL: unable to parse token: exp not satisfied Failed running "sql" ``` Validated error on setting wrong proxy params ``` ERROR: JWT authentication: unable to validate token SQLSTATE: 28000 Failed running "sql" ``` and logged error: ``` I240510 08:31:28.604141 1473 4@util/log/event_log.go:32 ⋮ [T1,Vsystem,n1,client=127.0.0.1:56289,hostssl,user=‹sourav.sarangi›] 3 ={"Timestamp":1715329888604122000,"EventType":"client_authentication_failed","InstanceID":1,"Network":"tcp","RemoteAddress":"‹127.0.0.1:56289›","SessionID":"17ce136f2a8ecd480000000000000001","Transport":"hostssl","User":"‹sourav.sarangi›","SystemIdentity":"‹sourav.sarangi›","Reason":"CREDENTIALS_INVALID","Detail":"JWT authentication: unable to validate token\nunable to fetch jwks: Get \"https://accounts.google.com/.well-known/openid-configuration\": proxyconnect tcp: dial tcp [::1]:3129: connect: connection refused","Method":"jwt_token"} ``` Verified access logs after setting up squid proxy and passing env HTTP_PROXY and HTTPS_PROXY params ``` 1715103871.761 144 ::1 TCP_TUNNEL/200 5708 CONNECT accounts.google.com:443 - HIER_DIRECT/74.125.200.84 - 1715103871.836 73 ::1 TCP_TUNNEL/200 5964 CONNECT www.googleapis.com:443 - HIER_DIRECT/142.250.182.10 - ``` fixes #123575, CRDB-38386, CRDB-38408 Epic None Release note: None
Describe the problem
The new JWKS Auto fetch feature introduced through #117054 ignored http transport env vars like HTTP_PROXY. As a result, the outbound connection to fetch jwks url would fail.
To Reproduce
Set
server.jwt_authentication.jwks_auto_fetch.enabled
cluster setting to True in an environment with HTTP_PROXY setup.Expected behavior
The JWT Auth should work as expected, but it returns error: "JWT authentication: unable to validate token"
Environment:
Additional context
This is similar to #32803
Jira issue: CRDB-38408
gz#21355
Related to : #123605
The text was updated successfully, but these errors were encountered: