Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

jwtauthccl : http client missing support for proxies #123575

Closed
BabuSrithar opened this issue May 3, 2024 · 3 comments · Fixed by #123697
Closed

jwtauthccl : http client missing support for proxies #123575

BabuSrithar opened this issue May 3, 2024 · 3 comments · Fixed by #123697
Assignees
Labels
C-bug Code not up to spec/doc, specs & docs deemed correct. Solution expected to change code/behavior. T-product-security

Comments

@BabuSrithar
Copy link
Contributor

BabuSrithar commented May 3, 2024

Describe the problem

The new JWKS Auto fetch feature introduced through #117054 ignored http transport env vars like HTTP_PROXY. As a result, the outbound connection to fetch jwks url would fail.

To Reproduce

Set server.jwt_authentication.jwks_auto_fetch.enabled cluster setting to True in an environment with HTTP_PROXY setup.

Expected behavior
The JWT Auth should work as expected, but it returns error: "JWT authentication: unable to validate token"

Environment:

  • CockroachDB version : v23.1.17, v24.1

Additional context
This is similar to #32803

Jira issue: CRDB-38408

gz#21355

Related to : #123605

@BabuSrithar BabuSrithar added the C-bug Code not up to spec/doc, specs & docs deemed correct. Solution expected to change code/behavior. label May 3, 2024
@BabuSrithar BabuSrithar self-assigned this May 3, 2024
@bdarnell
Copy link
Contributor

bdarnell commented May 3, 2024

This has happened a couple of times, and there are other inconsistencies in the HTTP clients we use (notably, the OIDC client library uses the default http transports which support proxies but don't have the lower timeouts we use elsewhere). We should try to centralize our use of http clients in 1 (or a few) places. This is tricky because some libraries access the default client directly. We need to either modify the global defaults to have the configuration we want (such as timeouts) or do something to disallow the use of global defaults (installing some sort of poisoned objects that will crash when used?)

@BabuSrithar
Copy link
Contributor Author

BabuSrithar commented May 4, 2024

This has happened a couple of times, and there are other inconsistencies in the HTTP clients we use (notably, the OIDC client library uses the default http transports which support proxies but don't have the lower timeouts we use elsewhere). We should try to centralize our use of http clients in 1 (or a few) places. This is tricky because some libraries access the default client directly. We need to either modify the global defaults to have the configuration we want (such as timeouts) or do something to disallow the use of global defaults (installing some sort of poisoned objects that will crash when used?)

Is there something we can do on lint / static check side? at least for direct HTTP client usage in our code.

souravcrl added a commit to souravcrl/cockroach that referenced this issue May 6, 2024
We are running into issues with jwt authentication and currently unable to
provide support as we are not logging the error from the http client used in the
authenticator. Thus, the PR looks to propagate the obtained error from
`ValidateJWTLogin`. We are also introducing `JWTClientTimeout` and
`JWTClientCustomCA` cluster settings such that these could be configured
directly for the http client used in authenticator. The http client now also
respects the system http proxy if set.

fixes cockroachdb#123575,
Epic CRDB-38386,

Release note(security update): We are adding 2 cluster settings
`server.jwt_authentication.client.timeout` and
`server.jwt_authentication.client.custom_ca` which can control the jwt auth
behaviour for http client calls.
souravcrl added a commit to souravcrl/cockroach that referenced this issue May 7, 2024
We are running into issues with jwt authentication and currently unable to
provide support as we are not logging the error from the http client used in the
authenticator. The PR looks to propagate this obtained error from
`ValidateJWTLogin` http client. We are also introducing `JWTClientTimeout` and
`JWTClientCustomCA` cluster settings such that these could be configured
directly for the http client used in authenticator. The http client now also
respects the system http proxy if set and has a cluster setting
`JWTClientSystemProxyEnabled` to enable it.

Validated the error details when presenting an expired token
```
ERROR: JWT authentication: invalid token
SQLSTATE: 28000
DETAIL: unable to parse token: exp not satisfied Failed running "sql"
```

Validated error on setting wrong proxy params
```
ERROR: JWT authentication: unable to validate token
SQLSTATE: 28000
DETAIL: unable to fetch jwks: Get
"https://accounts.google.com/.well-known/openid-configuration": proxyconnect
tcp: dial tcp [::1]:3129: connect: connection refused
Failed running "sql"
```

Verified access logs after setting up squid proxy and passing env HTTP_PROXY and
HTTPS_PROXY params
```
1715103871.761    144 ::1 TCP_TUNNEL/200 5708 CONNECT accounts.google.com:443 -
HIER_DIRECT/74.125.200.84 -
1715103871.836     73 ::1 TCP_TUNNEL/200 5964 CONNECT www.googleapis.com:443 -
HIER_DIRECT/142.250.182.10 -
```

fixes cockroachdb#123575,
Epic CRDB-38386,

Release note(security update): We are adding 3 cluster settings
`server.jwt_authentication.client.timeout`,
`server.jwt_authentication.client.custom_ca` and
`server.jwt_authentication.client.system_proxy.enabled` which can control the
jwt auth behaviour for http client calls.
@exalate-issue-sync exalate-issue-sync bot assigned souravcrl and unassigned BabuSrithar May 8, 2024
souravcrl added a commit to souravcrl/cockroach that referenced this issue May 8, 2024
We are running into issues with jwt authentication and currently unable to
provide support as we are not logging the error from the http client used in the
authenticator. The PR looks to propagate this obtained error from
`ValidateJWTLogin` http client. We are also introducing `JWTClientTimeout` and
`JWTClientCustomCA` cluster settings such that these could be configured
directly for the http client used in authenticator. The http client now also
respects the system http proxy if set and has a cluster setting
`JWTClientSystemProxyEnabled` to enable it.

Validated the error details when presenting an expired token
```
ERROR: JWT authentication: invalid token
SQLSTATE: 28000
DETAIL: unable to parse token: exp not satisfied Failed running "sql"
```

Validated error on setting wrong proxy params
```
ERROR: JWT authentication: unable to validate token
SQLSTATE: 28000
DETAIL: unable to fetch jwks: Get
"https://accounts.google.com/.well-known/openid-configuration": proxyconnect
tcp: dial tcp [::1]:3129: connect: connection refused
Failed running "sql"
```

Verified access logs after setting up squid proxy and passing env HTTP_PROXY and
HTTPS_PROXY params
```
1715103871.761    144 ::1 TCP_TUNNEL/200 5708 CONNECT accounts.google.com:443 -
HIER_DIRECT/74.125.200.84 -
1715103871.836     73 ::1 TCP_TUNNEL/200 5964 CONNECT www.googleapis.com:443 -
HIER_DIRECT/142.250.182.10 -
```

fixes cockroachdb#123575,
Epic CRDB-38386,

Release note(security update): We are adding 3 cluster settings
`server.jwt_authentication.client.timeout`,
`server.jwt_authentication.client.custom_ca` and
`server.jwt_authentication.client.system_proxy.enabled` which can control the
jwt auth behaviour for http client calls.
souravcrl added a commit to souravcrl/cockroach that referenced this issue May 9, 2024
We are running into issues with jwt authentication and currently unable to
provide support as we are not logging the error from the http client used in the
authenticator. The PR looks to propagate this obtained error from
`ValidateJWTLogin` http client. We are also introducing `JWTClientCustomCA`
cluster setting such that this could be configured directly for the http client
used in authenticator. The http client now also respects the system http proxy
if set.

Validated the error details when presenting an expired token
```
ERROR: JWT authentication: invalid token
SQLSTATE: 28000
DETAIL: unable to parse token: exp not satisfied Failed running "sql"
```

Validated error on setting wrong proxy params
```
ERROR: JWT authentication: unable to validate token
SQLSTATE: 28000
DETAIL: unable to fetch jwks: Get
"https://accounts.google.com/.well-known/openid-configuration": proxyconnect
tcp: dial tcp [::1]:3129: connect: connection refused
Failed running "sql"
```

Verified access logs after setting up squid proxy and passing env HTTP_PROXY and
HTTPS_PROXY params
```
1715103871.761    144 ::1 TCP_TUNNEL/200 5708 CONNECT accounts.google.com:443 -
HIER_DIRECT/74.125.200.84 -
1715103871.836     73 ::1 TCP_TUNNEL/200 5964 CONNECT www.googleapis.com:443 -
HIER_DIRECT/142.250.182.10 -
```

fixes cockroachdb#123575,
Epic CRDB-38386,

Release note(security update): We are adding a cluster settings
`server.jwt_authentication.client.custom_ca` which can tune the jwt auth
behaviour when we need custom ca http client calls.
souravcrl added a commit to souravcrl/cockroach that referenced this issue May 9, 2024
We are running into issues with jwt authentication and currently unable to
provide support as we are not logging the error from the http client used in the
authenticator. The PR looks to propagate this obtained error from
`ValidateJWTLogin` http client. We are also introducing `JWTClientCustomCA`
cluster setting such that this could be configured directly for the http client
used in authenticator. The http client now also respects the system http proxy
if set.

Validated the error details when presenting an expired token
```
ERROR: JWT authentication: invalid token
SQLSTATE: 28000
DETAIL: unable to parse token: exp not satisfied Failed running "sql"
```

Validated error on setting wrong proxy params
```
ERROR: JWT authentication: unable to validate token
SQLSTATE: 28000
DETAIL: unable to fetch jwks: Get
"https://accounts.google.com/.well-known/openid-configuration": proxyconnect
tcp: dial tcp [::1]:3129: connect: connection refused
Failed running "sql"
```

Verified access logs after setting up squid proxy and passing env HTTP_PROXY and
HTTPS_PROXY params
```
1715103871.761    144 ::1 TCP_TUNNEL/200 5708 CONNECT accounts.google.com:443 -
HIER_DIRECT/74.125.200.84 -
1715103871.836     73 ::1 TCP_TUNNEL/200 5964 CONNECT www.googleapis.com:443 -
HIER_DIRECT/142.250.182.10 -
```

fixes cockroachdb#123575,
Epic CRDB-38386,

Release note(security update): We are adding a cluster settings
`server.jwt_authentication.client.custom_ca` which can tune the jwt auth
behaviour when we need custom ca http client calls.
souravcrl added a commit to souravcrl/cockroach that referenced this issue May 10, 2024
We are running into issues with jwt authentication and currently unable to
provide support as we are not logging the error from the http client used in the
authenticator. The PR looks to propagate this obtained error from
`ValidateJWTLogin` http client. We are also introducing `JWTClientCustomCA`
cluster setting such that this could be configured directly for the http client
used in authenticator. The http client now also respects the system http proxy
if set.

Validated the error details when presenting an expired token
```
ERROR: JWT authentication: invalid token
SQLSTATE: 28000
DETAIL: unable to parse token: exp not satisfied Failed running "sql"
```

Validated error on setting wrong proxy params
```
ERROR: JWT authentication: unable to validate token
SQLSTATE: 28000
Failed running "sql"
```
and logged error:
```
I240510 08:31:28.604141 1473 4@util/log/event_log.go:32 ⋮
[T1,Vsystem,n1,client=127.0.0.1:56289,hostssl,user=‹sourav.sarangi›] 3
={"Timestamp":1715329888604122000,"EventType":"client_authentication_failed","InstanceID":1,"Network":"tcp","RemoteAddress":"‹127.0.0.1:56289›","SessionID":"17ce136f2a8ecd480000000000000001","Transport":"hostssl","User":"‹sourav.sarangi›","SystemIdentity":"‹sourav.sarangi›","Reason":"CREDENTIALS_INVALID","Detail":"JWT
authentication: unable to validate token\nunable to fetch jwks: Get
\"https://accounts.google.com/.well-known/openid-configuration\": proxyconnect
tcp: dial tcp [::1]:3129: connect: connection refused","Method":"jwt_token"}
```

Verified access logs after setting up squid proxy and passing env HTTP_PROXY and
HTTPS_PROXY params
```
1715103871.761    144 ::1 TCP_TUNNEL/200 5708 CONNECT accounts.google.com:443 -
HIER_DIRECT/74.125.200.84 -
1715103871.836     73 ::1 TCP_TUNNEL/200 5964 CONNECT www.googleapis.com:443 -
HIER_DIRECT/142.250.182.10 -
```

fixes cockroachdb#123575,
Epic CRDB-38386,

Release note(security update): We are adding a cluster settings
`server.jwt_authentication.client.custom_ca` which can tune the jwt auth
behaviour when we need custom ca http client calls.
souravcrl added a commit to souravcrl/cockroach that referenced this issue May 10, 2024
We are running into issues with jwt authentication and currently unable to
provide support as we are not logging the error from the http client used in the
authenticator. The PR looks to propagate this obtained error from
`ValidateJWTLogin` http client. We are also introducing `JWTClientCustomCA`
cluster setting such that this could be configured directly for the http client
used in authenticator. The http client now also respects the system http proxy
if set.

Validated the error details when presenting an expired token
```
ERROR: JWT authentication: invalid token
SQLSTATE: 28000
DETAIL: unable to parse token: exp not satisfied Failed running "sql"
```

Validated error on setting wrong proxy params
```
ERROR: JWT authentication: unable to validate token
SQLSTATE: 28000
Failed running "sql"
```
and logged error:
```
I240510 08:31:28.604141 1473 4@util/log/event_log.go:32 ⋮
[T1,Vsystem,n1,client=127.0.0.1:56289,hostssl,user=‹sourav.sarangi›] 3
={"Timestamp":1715329888604122000,"EventType":"client_authentication_failed","InstanceID":1,"Network":"tcp","RemoteAddress":"‹127.0.0.1:56289›","SessionID":"17ce136f2a8ecd480000000000000001","Transport":"hostssl","User":"‹sourav.sarangi›","SystemIdentity":"‹sourav.sarangi›","Reason":"CREDENTIALS_INVALID","Detail":"JWT
authentication: unable to validate token\nunable to fetch jwks: Get
\"https://accounts.google.com/.well-known/openid-configuration\": proxyconnect
tcp: dial tcp [::1]:3129: connect: connection refused","Method":"jwt_token"}
```

Verified access logs after setting up squid proxy and passing env HTTP_PROXY and
HTTPS_PROXY params
```
1715103871.761    144 ::1 TCP_TUNNEL/200 5708 CONNECT accounts.google.com:443 -
HIER_DIRECT/74.125.200.84 -
1715103871.836     73 ::1 TCP_TUNNEL/200 5964 CONNECT www.googleapis.com:443 -
HIER_DIRECT/142.250.182.10 -
```

fixes cockroachdb#123575,
Epic CRDB-38386, CRDB-38408

Release note(security update): We are adding a cluster settings
`server.jwt_authentication.client.custom_ca` which can tune the jwt auth
behaviour when we need custom ca http client calls.
souravcrl added a commit to souravcrl/cockroach that referenced this issue May 13, 2024
We are running into issues with jwt authentication and currently unable to
provide support as we are not logging the error from the http client used in the
authenticator. The PR looks to propagate this obtained error from
`ValidateJWTLogin` http client. We are also introducing `JWTClientCustomCA`
cluster setting such that this could be configured directly for the http client
used in authenticator. The http client now also respects the system http proxy
if set.

Validated the error details when presenting an expired token
```
ERROR: JWT authentication: invalid token
SQLSTATE: 28000
DETAIL: unable to parse token: exp not satisfied Failed running "sql"
```

Validated error on setting wrong proxy params
```
ERROR: JWT authentication: unable to validate token
SQLSTATE: 28000
Failed running "sql"
```
and logged error:
```
I240510 08:31:28.604141 1473 4@util/log/event_log.go:32 ⋮
[T1,Vsystem,n1,client=127.0.0.1:56289,hostssl,user=‹sourav.sarangi›] 3
={"Timestamp":1715329888604122000,"EventType":"client_authentication_failed","InstanceID":1,"Network":"tcp","RemoteAddress":"‹127.0.0.1:56289›","SessionID":"17ce136f2a8ecd480000000000000001","Transport":"hostssl","User":"‹sourav.sarangi›","SystemIdentity":"‹sourav.sarangi›","Reason":"CREDENTIALS_INVALID","Detail":"JWT
authentication: unable to validate token\nunable to fetch jwks: Get
\"https://accounts.google.com/.well-known/openid-configuration\": proxyconnect
tcp: dial tcp [::1]:3129: connect: connection refused","Method":"jwt_token"}
```

Verified access logs after setting up squid proxy and passing env HTTP_PROXY and
HTTPS_PROXY params
```
1715103871.761    144 ::1 TCP_TUNNEL/200 5708 CONNECT accounts.google.com:443 -
HIER_DIRECT/74.125.200.84 -
1715103871.836     73 ::1 TCP_TUNNEL/200 5964 CONNECT www.googleapis.com:443 -
HIER_DIRECT/142.250.182.10 -
```

fixes cockroachdb#123575, CRDB-38386, CRDB-38408
Epic None

Release note(security update): We are adding a cluster settings
`server.jwt_authentication.client.custom_ca` which can tune the jwt auth
behaviour when we need custom ca http client calls.
souravcrl added a commit to souravcrl/cockroach that referenced this issue May 13, 2024
We are running into issues with jwt authentication and currently unable to
provide support as we are not logging the error from the http client used in the
authenticator. The PR looks to propagate this obtained error from
`ValidateJWTLogin` http client. The http client now also respects the system
http proxy if set.

Validated the error details when presenting an expired token
```
ERROR: JWT authentication: invalid token
SQLSTATE: 28000
DETAIL: unable to parse token: exp not satisfied Failed running "sql"
```

Validated error on setting wrong proxy params
```
ERROR: JWT authentication: unable to validate token
SQLSTATE: 28000
Failed running "sql"
```
and logged error:
```
I240510 08:31:28.604141 1473 4@util/log/event_log.go:32 ⋮
[T1,Vsystem,n1,client=127.0.0.1:56289,hostssl,user=‹sourav.sarangi›] 3
={"Timestamp":1715329888604122000,"EventType":"client_authentication_failed","InstanceID":1,"Network":"tcp","RemoteAddress":"‹127.0.0.1:56289›","SessionID":"17ce136f2a8ecd480000000000000001","Transport":"hostssl","User":"‹sourav.sarangi›","SystemIdentity":"‹sourav.sarangi›","Reason":"CREDENTIALS_INVALID","Detail":"JWT
authentication: unable to validate token\nunable to fetch jwks: Get
\"https://accounts.google.com/.well-known/openid-configuration\": proxyconnect
tcp: dial tcp [::1]:3129: connect: connection refused","Method":"jwt_token"}
```

Verified access logs after setting up squid proxy and passing env HTTP_PROXY and
HTTPS_PROXY params
```
1715103871.761    144 ::1 TCP_TUNNEL/200 5708 CONNECT accounts.google.com:443 -
HIER_DIRECT/74.125.200.84 -
1715103871.836     73 ::1 TCP_TUNNEL/200 5964 CONNECT www.googleapis.com:443 -
HIER_DIRECT/142.250.182.10 -
```

fixes cockroachdb#123575, CRDB-38386, CRDB-38408
Epic None

Release note: None
craig bot pushed a commit that referenced this issue May 14, 2024
123697: ccl,sql,util: Fix jwt auth and add sensitive error logs r=souravcrl a=souravcrl

ccl,sql,util: Fix jwt auth and add sensitive error logs

We are running into issues with jwt authentication and currently unable to
provide support as we are not logging the error from the http client used in the
authenticator. The PR looks to propagate this obtained error from
`ValidateJWTLogin` http client. The http client now also respects the system
http proxy if set.

Validated the error details when presenting an expired token
```
ERROR: JWT authentication: invalid token
SQLSTATE: 28000
DETAIL: unable to parse token: exp not satisfied Failed running "sql"
```

Validated error on setting wrong proxy params
```
ERROR: JWT authentication: unable to validate token
SQLSTATE: 28000
Failed running "sql"
```
and logged error:
```
I240510 08:31:28.604141 1473 4@util/log/event_log.go:32 ⋮
[T1,Vsystem,n1,client=127.0.0.1:56289,hostssl,user=‹sourav.sarangi›] 3
={"Timestamp":1715329888604122000,"EventType":"client_authentication_failed","InstanceID":1,"Network":"tcp","RemoteAddress":"‹127.0.0.1:56289›","SessionID":"17ce136f2a8ecd480000000000000001","Transport":"hostssl","User":"‹sourav.sarangi›","SystemIdentity":"‹sourav.sarangi›","Reason":"CREDENTIALS_INVALID","Detail":"JWT
authentication: unable to validate token\nunable to fetch jwks: Get
\"https://accounts.google.com/.well-known/openid-configuration\": proxyconnect
tcp: dial tcp [::1]:3129: connect: connection refused","Method":"jwt_token"}
```

Verified access logs after setting up squid proxy and passing env HTTP_PROXY and
HTTPS_PROXY params
```
1715103871.761    144 ::1 TCP_TUNNEL/200 5708 CONNECT accounts.google.com:443 -
HIER_DIRECT/74.125.200.84 -
1715103871.836     73 ::1 TCP_TUNNEL/200 5964 CONNECT www.googleapis.com:443 -
HIER_DIRECT/142.250.182.10 -
```

fixes #123575, CRDB-38386, CRDB-38408
Epic None

Release note: Noneccl,sql,util: Fix jwt auth and add sensitive error logs

We are running into issues with jwt authentication and currently unable to
provide support as we are not logging the error from the http client used in the
authenticator. The PR looks to propagate this obtained error from
`ValidateJWTLogin` http client. The http client now also respects the system
http proxy if set.

Validated the error details when presenting an expired token
```
ERROR: JWT authentication: invalid token
SQLSTATE: 28000
DETAIL: unable to parse token: exp not satisfied Failed running "sql"
```

Validated error on setting wrong proxy params
```
ERROR: JWT authentication: unable to validate token
SQLSTATE: 28000
Failed running "sql"
```
and logged error:
```
I240510 08:31:28.604141 1473 4@util/log/event_log.go:32 ⋮
[T1,Vsystem,n1,client=127.0.0.1:56289,hostssl,user=‹sourav.sarangi›] 3
={"Timestamp":1715329888604122000,"EventType":"client_authentication_failed","InstanceID":1,"Network":"tcp","RemoteAddress":"‹127.0.0.1:56289›","SessionID":"17ce136f2a8ecd480000000000000001","Transport":"hostssl","User":"‹sourav.sarangi›","SystemIdentity":"‹sourav.sarangi›","Reason":"CREDENTIALS_INVALID","Detail":"JWT
authentication: unable to validate token\nunable to fetch jwks: Get
\"https://accounts.google.com/.well-known/openid-configuration\": proxyconnect
tcp: dial tcp [::1]:3129: connect: connection refused","Method":"jwt_token"}
```

Verified access logs after setting up squid proxy and passing env HTTP_PROXY and
HTTPS_PROXY params
```
1715103871.761    144 ::1 TCP_TUNNEL/200 5708 CONNECT accounts.google.com:443 -
HIER_DIRECT/74.125.200.84 -
1715103871.836     73 ::1 TCP_TUNNEL/200 5964 CONNECT www.googleapis.com:443 -
HIER_DIRECT/142.250.182.10 -
```

fixes #123575, CRDB-38386, CRDB-38408
Epic None

Release note: None

Co-authored-by: Sourav Sarangi <[email protected]>
@craig craig bot closed this as completed in dadee6d May 14, 2024
souravcrl added a commit to souravcrl/cockroach that referenced this issue May 15, 2024
We are running into issues with jwt authentication and currently unable to
provide support as we are not logging the error from the http client used in the
authenticator. The PR looks to propagate this obtained error from
`ValidateJWTLogin` http client. The http client now also respects the system
http proxy if set.

Validated the error details when presenting an expired token
```
ERROR: JWT authentication: invalid token
SQLSTATE: 28000
DETAIL: unable to parse token: exp not satisfied Failed running "sql"
```

Validated error on setting wrong proxy params
```
ERROR: JWT authentication: unable to validate token
SQLSTATE: 28000
Failed running "sql"
```
and logged error:
```
I240510 08:31:28.604141 1473 4@util/log/event_log.go:32 ⋮
[T1,Vsystem,n1,client=127.0.0.1:56289,hostssl,user=‹sourav.sarangi›] 3
={"Timestamp":1715329888604122000,"EventType":"client_authentication_failed","InstanceID":1,"Network":"tcp","RemoteAddress":"‹127.0.0.1:56289›","SessionID":"17ce136f2a8ecd480000000000000001","Transport":"hostssl","User":"‹sourav.sarangi›","SystemIdentity":"‹sourav.sarangi›","Reason":"CREDENTIALS_INVALID","Detail":"JWT
authentication: unable to validate token\nunable to fetch jwks: Get
\"https://accounts.google.com/.well-known/openid-configuration\": proxyconnect
tcp: dial tcp [::1]:3129: connect: connection refused","Method":"jwt_token"}
```

Verified access logs after setting up squid proxy and passing env HTTP_PROXY and
HTTPS_PROXY params
```
1715103871.761    144 ::1 TCP_TUNNEL/200 5708 CONNECT accounts.google.com:443 -
HIER_DIRECT/74.125.200.84 -
1715103871.836     73 ::1 TCP_TUNNEL/200 5964 CONNECT www.googleapis.com:443 -
HIER_DIRECT/142.250.182.10 -
```

fixes cockroachdb#123575, CRDB-38386, CRDB-38408
Epic None

Release note: None
souravcrl added a commit to souravcrl/cockroach that referenced this issue May 15, 2024
We are running into issues with jwt authentication and currently unable to
provide support as we are not logging the error from the http client used in the
authenticator. The PR looks to propagate this obtained error from
`ValidateJWTLogin` http client. The http client now also respects the system
http proxy if set.

Validated the error details when presenting an expired token
```
ERROR: JWT authentication: invalid token
SQLSTATE: 28000
DETAIL: unable to parse token: exp not satisfied Failed running "sql"
```

Validated error on setting wrong proxy params
```
ERROR: JWT authentication: unable to validate token
SQLSTATE: 28000
Failed running "sql"
```
and logged error:
```
I240510 08:31:28.604141 1473 4@util/log/event_log.go:32 ⋮
[T1,Vsystem,n1,client=127.0.0.1:56289,hostssl,user=‹sourav.sarangi›] 3
={"Timestamp":1715329888604122000,"EventType":"client_authentication_failed","InstanceID":1,"Network":"tcp","RemoteAddress":"‹127.0.0.1:56289›","SessionID":"17ce136f2a8ecd480000000000000001","Transport":"hostssl","User":"‹sourav.sarangi›","SystemIdentity":"‹sourav.sarangi›","Reason":"CREDENTIALS_INVALID","Detail":"JWT
authentication: unable to validate token\nunable to fetch jwks: Get
\"https://accounts.google.com/.well-known/openid-configuration\": proxyconnect
tcp: dial tcp [::1]:3129: connect: connection refused","Method":"jwt_token"}
```

Verified access logs after setting up squid proxy and passing env HTTP_PROXY and
HTTPS_PROXY params
```
1715103871.761    144 ::1 TCP_TUNNEL/200 5708 CONNECT accounts.google.com:443 -
HIER_DIRECT/74.125.200.84 -
1715103871.836     73 ::1 TCP_TUNNEL/200 5964 CONNECT www.googleapis.com:443 -
HIER_DIRECT/142.250.182.10 -
```

fixes cockroachdb#123575, CRDB-38386, CRDB-38408
Epic None

Release note: None
souravcrl added a commit that referenced this issue May 15, 2024
We are running into issues with jwt authentication and currently unable to
provide support as we are not logging the error from the http client used in the
authenticator. The PR looks to propagate this obtained error from
`ValidateJWTLogin` http client. The http client now also respects the system
http proxy if set.

Validated the error details when presenting an expired token
```
ERROR: JWT authentication: invalid token
SQLSTATE: 28000
DETAIL: unable to parse token: exp not satisfied Failed running "sql"
```

Validated error on setting wrong proxy params
```
ERROR: JWT authentication: unable to validate token
SQLSTATE: 28000
Failed running "sql"
```
and logged error:
```
I240510 08:31:28.604141 1473 4@util/log/event_log.go:32 ⋮
[T1,Vsystem,n1,client=127.0.0.1:56289,hostssl,user=‹sourav.sarangi›] 3
={"Timestamp":1715329888604122000,"EventType":"client_authentication_failed","InstanceID":1,"Network":"tcp","RemoteAddress":"‹127.0.0.1:56289›","SessionID":"17ce136f2a8ecd480000000000000001","Transport":"hostssl","User":"‹sourav.sarangi›","SystemIdentity":"‹sourav.sarangi›","Reason":"CREDENTIALS_INVALID","Detail":"JWT
authentication: unable to validate token\nunable to fetch jwks: Get
\"https://accounts.google.com/.well-known/openid-configuration\": proxyconnect
tcp: dial tcp [::1]:3129: connect: connection refused","Method":"jwt_token"}
```

Verified access logs after setting up squid proxy and passing env HTTP_PROXY and
HTTPS_PROXY params
```
1715103871.761    144 ::1 TCP_TUNNEL/200 5708 CONNECT accounts.google.com:443 -
HIER_DIRECT/74.125.200.84 -
1715103871.836     73 ::1 TCP_TUNNEL/200 5964 CONNECT www.googleapis.com:443 -
HIER_DIRECT/142.250.182.10 -
```

fixes #123575, CRDB-38386, CRDB-38408
Epic None

Release note: None
aadityasondhi pushed a commit to aadityasondhi/cockroach that referenced this issue May 15, 2024
We are running into issues with jwt authentication and currently unable to
provide support as we are not logging the error from the http client used in the
authenticator. The PR looks to propagate this obtained error from
`ValidateJWTLogin` http client. The http client now also respects the system
http proxy if set.

Validated the error details when presenting an expired token
```
ERROR: JWT authentication: invalid token
SQLSTATE: 28000
DETAIL: unable to parse token: exp not satisfied Failed running "sql"
```

Validated error on setting wrong proxy params
```
ERROR: JWT authentication: unable to validate token
SQLSTATE: 28000
Failed running "sql"
```
and logged error:
```
I240510 08:31:28.604141 1473 4@util/log/event_log.go:32 ⋮
[T1,Vsystem,n1,client=127.0.0.1:56289,hostssl,user=‹sourav.sarangi›] 3
={"Timestamp":1715329888604122000,"EventType":"client_authentication_failed","InstanceID":1,"Network":"tcp","RemoteAddress":"‹127.0.0.1:56289›","SessionID":"17ce136f2a8ecd480000000000000001","Transport":"hostssl","User":"‹sourav.sarangi›","SystemIdentity":"‹sourav.sarangi›","Reason":"CREDENTIALS_INVALID","Detail":"JWT
authentication: unable to validate token\nunable to fetch jwks: Get
\"https://accounts.google.com/.well-known/openid-configuration\": proxyconnect
tcp: dial tcp [::1]:3129: connect: connection refused","Method":"jwt_token"}
```

Verified access logs after setting up squid proxy and passing env HTTP_PROXY and
HTTPS_PROXY params
```
1715103871.761    144 ::1 TCP_TUNNEL/200 5708 CONNECT accounts.google.com:443 -
HIER_DIRECT/74.125.200.84 -
1715103871.836     73 ::1 TCP_TUNNEL/200 5964 CONNECT www.googleapis.com:443 -
HIER_DIRECT/142.250.182.10 -
```

fixes cockroachdb#123575, CRDB-38386, CRDB-38408
Epic None

Release note: None
blathers-crl bot pushed a commit that referenced this issue May 17, 2024
We are running into issues with jwt authentication and currently unable to
provide support as we are not logging the error from the http client used in the
authenticator. The PR looks to propagate this obtained error from
`ValidateJWTLogin` http client. The http client now also respects the system
http proxy if set.

Validated the error details when presenting an expired token
```
ERROR: JWT authentication: invalid token
SQLSTATE: 28000
DETAIL: unable to parse token: exp not satisfied Failed running "sql"
```

Validated error on setting wrong proxy params
```
ERROR: JWT authentication: unable to validate token
SQLSTATE: 28000
Failed running "sql"
```
and logged error:
```
I240510 08:31:28.604141 1473 4@util/log/event_log.go:32 ⋮
[T1,Vsystem,n1,client=127.0.0.1:56289,hostssl,user=‹sourav.sarangi›] 3
={"Timestamp":1715329888604122000,"EventType":"client_authentication_failed","InstanceID":1,"Network":"tcp","RemoteAddress":"‹127.0.0.1:56289›","SessionID":"17ce136f2a8ecd480000000000000001","Transport":"hostssl","User":"‹sourav.sarangi›","SystemIdentity":"‹sourav.sarangi›","Reason":"CREDENTIALS_INVALID","Detail":"JWT
authentication: unable to validate token\nunable to fetch jwks: Get
\"https://accounts.google.com/.well-known/openid-configuration\": proxyconnect
tcp: dial tcp [::1]:3129: connect: connection refused","Method":"jwt_token"}
```

Verified access logs after setting up squid proxy and passing env HTTP_PROXY and
HTTPS_PROXY params
```
1715103871.761    144 ::1 TCP_TUNNEL/200 5708 CONNECT accounts.google.com:443 -
HIER_DIRECT/74.125.200.84 -
1715103871.836     73 ::1 TCP_TUNNEL/200 5964 CONNECT www.googleapis.com:443 -
HIER_DIRECT/142.250.182.10 -
```

fixes #123575, CRDB-38386, CRDB-38408
Epic None

Release note: None
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
C-bug Code not up to spec/doc, specs & docs deemed correct. Solution expected to change code/behavior. T-product-security
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants