CFP: Support basic-auth/oauth2/mTLS for north-south traffic (GatewayAPI/Ingress)

[chancez]

Cilium Feature Proposal

Describe the feature you'd like

I'd like to be able to expose my applications with various forms of authentication. For example, I want to expose hubble-relay outside of my cluster.

Various authentication mechanisms work better for various situations. Generally all 3 mentioned should be relatively standard APIs for configuration. One thing that would be really nice, is to be able to configure different authentication mechanisms based on different rules. For example, mTLS/client-certs if the request is coming from a certain IP range, and basic-auth/oauth2 if a specific user-agent is set.

(Optional) Describe your proposed solution

I would like these to be expose in GatewayAPI, but Ingress is fine too.

[artyom-p]

Planning to migrate from aws-cni + istio to cilium, we have external auth middleware on istios ingressgateway which redirects user to our keykloak instance, at the moment absence of this feature is one of the blockers for migration.

[youngnick]

@artyom-p, could you point me to any documentation about that middleware? The work to design something to do this for Gateway API is still pretty early, so the more concrete examples we have, the better.

I agree these should be exposed in Gateway API, currently Policy Attachment is the way to do this, because noone is sure of the right way to describe the config.

tl;dr Upstream Gateway API design work needs to happen before we can implement this for Gateway API. I'd prefer to wait for that than to add a whole bunch of annotations to Ingress (annotations are sticky and unstructured, which is a bad combination for supportability.)

[artyom-p]

Thats not related to Gateway API, its a feature of Istio/Envoy itself, just a middleware. You can check example here.

[chancez]

Yeah, that filter would be pretty useful. It's pretty common to avoid double proxying with an auth proxy, and using the auth proxy to act more as an authentication/authorization API. This is often used with oauth2-proxy/authelia. I agree adding support for that would be very useful for a lot of users.

[youngnick]

Ah, okay, that's using the ext_auth support, I agree that's very useful, thanks.

[github-actions]

This issue has been automatically marked as stale because it has not
had recent activity. It will be closed if no further activity occurs.

[github-actions]

This issue has not seen any activity since it was marked stale.
Closing.

[laibe]

would be very much interested in this feature as well!

[toms-place]

+1 for this feature! (:

[jbgcarnes]

Very interested as well.

[rauanmayemir]

Now that cilium supports running envoy in a separate DS, it would make sense to include more built-in extensions like ext_authz, etc.

[Jamison1]

I am looking for this as well.

Upstream GatewayAPI has an issue tracker for auth. Its listed as backlog due to needing help on it.

As it may be some time before upstream GatewayAPI is resolved, maybe something simple can be enabled in the meantime.

[from-nibly]

Also a blocker for us to move to cilium ingress.

[unusualfor]

This is preventing us to fully move to Cilium as well

[liyihuang]

Is it possible to have the field under network policy's HTTP for jwt authz when the gateway API does not have the API? network policy can also be used between backend services, that's useful when the backend svc using jwt to do the authz

[EternalDeiwos]

There was an excellent proposal in kubernetes-sigs/gateway-api#532 that would have allowed for authentication/authorization to be delegated to some external service like OPA. Ideally Ingress, Gateway (global auth) and HTTPRoute could each have something like this.

[Jamison1]

Here is an example of GatewayAPI auth using an API key as a reference as it seems the Kubernetes upstream is still listed as help needed.

[gsharma-jiggzy]

Not sure if anyone looked at the Envoy Gateway (https://gateway.envoyproxy.io/latest/user/oidc/) but it has a concept called a security policy for oidc in the next release. It would be great if the security policy would be implemented in a similar manner.

[kevin-shelaga]

+1 to this feature!

[lukapetrovic-git]

definite +1 for this!

[abctaylor]

+1

[aldycool]

+1

[svilcu]

+1

[simonostendorf]

This is needed to switch to cilium service mesh.

[ormandj]

For the mTLS side: https://gateway-api.sigs.k8s.io/geps/gep-91/

[youngnick]

Auth for north-south Gateway API connections is still being discussed, it looks likely that we'll try and push something forward in Gateway API 1.3, which will start once Gateway 1.2 is out (around Kubecon NA 2024 in mid-November).

Just wanted to let everyone know that this issue is not forgotten, we definitely want to get to it, but we need to work with upstream on doing it the right way.

[youngnick]

Oh, I should note that GEP-91 that @ormandj mentioned above is for specifying a client certificate, but it really only guarantees that the client certificate is signed by the specified CA, it can't handle any authorization based on the contents of the certificate or JWT or anything - that will require something else (which is what we're going to work on soon).

[ormandj]

@youngnick good point - this has been a major sticking point for us with moving towards Gateway API for everything; how to not just authenticate clients but handle authorization based on their identity.

As a temporary solution, you can create and manage a CA per role and generate client certificates this way, but the management overhead isn't ideal. It'd be nice if there was something available like this, at least as a temporary stop-gap until the more correct/full-featured implementation was in place.

[zviratko]

It would be great if one could add a header containing the client certificate DN (and possibly the CA chain that authenticated it). That way many applications can be extended to consume this (as long as that header can be trusted that is).

[AlexanderSarson]

+1

[youngnick]

The Gateway API Auth GEP is scheduled to be worked on as part of the v1.3 release, see kubernetes-sigs/gateway-api#1494 for the details.

Once this GEP has moved to Experimental, we will add support for it in Cilium. I can't guarantee that will happen in the Cilium 1.17 timeframe, but we will definitely add it once it's in upstream.