fix: translator reports errors for existing clusters and secretes

[zhaohuabing]

Fixes #4706 xDS translation failed when oidc tokenEndpoint and jwt remoteJWKS are specified within the same security policy and using the same hostname

Refactor: skips adding the cluster/secrets and returns nil to make the code cleaner and easier to maintain. It's safe to remove ErrXdsClusterExists and ErrXdsSecretsExists as they don't need to be handled in any places.

Release Notes: Yes

Test before the fix:

--- FAIL: TestTranslateXds (0.43s)
    --- FAIL: TestTranslateXds/securitypolicy-with-oidc-jwt-authz (0.00s)
        translator_test.go:142: securitypolicy-with-oidc-jwt-authz
        translator_test.go:143: 
                Error Trace:    /home/ubuntu/gateway/internal/xds/translator/translator_test.go:143
                Error:          Received unexpected error:
                                xds cluster exists
                Test:           TestTranslateXds/securitypolicy-with-oidc-jwt-authz
FAIL
FAIL    github.com/envoyproxy/gateway/internal/xds/translator   0.673s
FAIL

After:

ok      github.com/envoyproxy/gateway/internal/xds/translator   0.251s

[codecov]

Codecov Report

Attention: Patch coverage is 72.22222% with 10 lines in your changes missing coverage. Please review.

Project coverage is 65.65%. Comparing base (f99c36c) to head (2759831).
Report is 1 commits behind head on main.

Files with missing lines	Patch %	Lines
internal/xds/translator/translator.go	72.72%	1 Missing and 2 partials ⚠️
internal/gatewayapi/securitypolicy.go	66.66%	0 Missing and 2 partials ⚠️
internal/xds/translator/extauth.go	0.00%	0 Missing and 2 partials ⚠️
internal/xds/translator/accesslog.go	50.00%	0 Missing and 1 partial ⚠️
internal/xds/translator/extproc.go	0.00%	0 Missing and 1 partial ⚠️
internal/xds/translator/oidc.go	50.00%	0 Missing and 1 partial ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #4707      +/-   ##
==========================================
+ Coverage   65.63%   65.65%   +0.02%     
==========================================
  Files         211      211              
  Lines       32017    31996      -21     
==========================================
- Hits        21014    21008       -6     
+ Misses       9761     9751      -10     
+ Partials     1242     1237       -5     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

---

🚨 Try these New Features:

• Flaky Tests Detection - Detect and resolve failed and flaky tests

[arkodg]

hey @zhaohuabing is the issue that we are generating the same name for the jwks and oidc clusters if both are set in the policy ? shouldnt we be using an additional string value to differentiate them ?

[zhaohuabing]

hey @zhaohuabing is the issue that we are generating the same name for the jwks and oidc clusters if both are set in the policy ? shouldnt we be using an additional string value to differentiate them ?

For the cluster generated by a url, EG uses the host and port for the cluster name to avoid creating duplicated clusters for the same host+port combination.

jwt: 
 providers:
   - remoteJWKS:
       uri: https://oidc.example.com/auth/realms/example/protocol/openid-connect/cert

oidc:
 provider:
   tokenEndpoint: https://oidc.example.com/oauth/token

The name of the generated cluster: oidc_example_com_443.

We could change this logic to generate an unique name for each single oidc or jwt configuration. However, we should also ensure that the translator shouldn't throw error if the cluster already exists.

[arkodg]

hey @zhaohuabing is the issue that we are generating the same name for the jwks and oidc clusters if both are set in the policy ? shouldnt we be using an additional string value to differentiate them ?

For the cluster generated by a url, EG uses the host and port for the cluster name to avoid creating duplicated clusters for the same host+port combination.

jwt: 
 providers:
   - remoteJWKS:
       uri: https://oidc.example.com/auth/realms/example/protocol/openid-connect/cert

oidc:
 provider:
   tokenEndpoint: https://oidc.example.com/oauth/token

The name of the generated cluster: oidc_example_com_443.

We could change this logic to generate an unique name for each single oidc or jwt configuration. However, we should also ensure that the translator shouldn't throw error if the cluster already exists.

is it safe to reuse the same cluster configuration ? is the naming different when use the backendRefs field ?

[zhaohuabing]

is it safe to reuse the same cluster configuration ? is the naming different when use the backendRefs field ?

It's safe to reuse the asme cluster for ulr generated cluster as the cluster confiugration is identical for the same host+port combination.

For OIDC provider with backendRefs, EG generate an unique cluster name like securitypolicy/envoy-gateway/policy-for-gateway/0

[zhaohuabing]

For OIDC provider with backendRefs, EG generate an unique cluster name like securitypolicy/envoy-gateway/policy-for-gateway/0.

Ha, there is also a bug here, the index is always 0, which generates duplicated name for different clusters if there're both ext auth and oidc whithin a SecurityPolicy. Will fix it in this PR as well.

[arkodg]

For OIDC provider with backendRefs, EG generate an unique cluster name like securitypolicy/envoy-gateway/policy-for-gateway/0.

Ha, there is also a bug here, the index is always 0, which generates duplicated name for different clusters if there're both ext auth and oidc whithin a SecurityPolicy. Will fix it in this PR as well.

nice catch, prob needs another prefix like jwt, oidc after policy name

[arkodg]

why is the index needed ?

[zhaohuabing]

A cluster is created for each parent, so an index is added to the cluster name to avoid name conflict.

var (
		extAutClusterIndex = 0
		oidcClusterIndex   = 0
	)
	for _, p := range parentRefs {
		parentRefCtx := GetRouteParentContext(route, p)
		gtwCtx := parentRefCtx.GetGateway()
		if gtwCtx == nil {
			continue
		}

		var extAuth *ir.ExtAuth
		if policy.Spec.ExtAuth != nil {
			if extAuth, err = t.buildExtAuth(
				policy,
				resources,
				gtwCtx.envoyProxy,
				extAutClusterIndex,
			); err != nil {
				err = perr.WithMessage(err, "ExtAuth")
				errs = errors.Join(errs, err)
			}
			extAutClusterIndex++
		}

		var oidc *ir.OIDC
		if policy.Spec.OIDC != nil {
			if oidc, err = t.buildOIDC(
				policy,
				resources,
				gtwCtx.envoyProxy, // TODO zhaohuabing: Only the last EnvoyProxy will be used as the OIDC name doesn't include the cluster index
				oidcClusterIndex,
			); err != nil {
				err = perr.WithMessage(err, "OIDC")
				errs = errors.Join(errs, err)
			}
			oidcClusterIndex++
		}
               ...
	}
	return errs
}

[arkodg]

It's the same policy though, can't the clusters be reused ?

[zhaohuabing]

It's because the EnvoyProxy configuration is different for each parent.

There's still an issue here, only the last one will be applied as the OIDC name is identical, I've added a TODO and will address this later in a follow-up PR.

#4707 (comment)

[arkodg]

if its due to envoy proxy resource attributes that make the cluster unique, then lets append /<ns>-<name>/ of EP instead to maximize reuse ?

[arkodg]

or listener name, the index here is not used to debug and is not useful for reuse

[zhaohuabing]

Rethinking on this, since EnvoyProxy is an optional parameter, its name may not be suitable to be used as part of the generated cluster name. We can always use EndpointRoutingType or ServiceRoutingType for OIDC, ExtProc, and other none-route backend clusters and don't need to use the RoutingType in the EnvoyProxy.

@arkodg Could we address this in a follow-up PR for #4720? This is a minor issue and won't break OIDC. Addressing it in a separate PR will make cherry-picking to v1.2.2 easier.

[arkodg]

@zhaohuabing I'd prefer if we removed the index altogether and raised another GH issue for dealing with policies targeting routes linked to multiple envoy proxies that are impacting the upstream TLS.

[arkodg]

LGTM thanks !