SecurityPolicy with JWT extractFromCookies and CORSenabled fails with 401

[apjoseph]

Description:
The following SecurityPolicy configuration results in 401 errors on OPTIONS requests; SPAs must use http-only cookies and include credentials in requests. However credentials won't be sent until Access-Control-Allow-Credentials"true is returned in the response headers. This results in perpetual denial of requests.

  jwt:
    providers:
      - name: auth0
        extractFrom:
          cookies:
            - BearerToken
        remoteJWKS:
          uri: https://web.example.com/.well-known/jwks.json

Environment:
0.0.0-latest
https://hub.docker.com/layers/envoyproxy/gateway-helm/v0.0.0-latest/images/sha256-246e6cd244ec3a6d6db37acf9a1aba80a4599a6630aa1be40e24875843c7a780?context=explore

[apjoseph]

As an attempted workaround I created another route that matches OPTIONS specifically

        matches:
          - path:
              type: PathPrefix
              value: /
            method: OPTIONS
        filters:
          - type: ResponseHeaderModifier
            responseHeaderModifier:
              set:
                - name: "access-control-allow-credentials"
                  value: "true"

This ALMOST works. Unfortunately it appears that when envoy handles an options request , it completely ignores the ResponseHeaderModifier

[zirain]

cc @zhaohuabing

[apjoseph]

As an update I also tried manually appending this via an EnvoyPatchPolicy which failed with "virtualHosts already exists"
From my testing, it appears that the patch policy is applied separately from the securitypolicy rather than being merged into it.
I'd expect the PatchPolicies to be applied AFTER all *Route and SecurityPolicies have been applied and then merged into them.

I'm pretty new to Envoy/EnvoyGateway, but It looks like right now to make a simple fix to a security policy, like adding the missing allowCredentials option, I'd basically have to rewrite the routeConfig for the entire https listener and write out every aspect of my routes via xDS -meaning I couldn't apply any securitypolicies or *route CRDs without causing a conflict.

If this is the case I find this a bit too inflexible for my needs. In generally I'm pretty pleased with this Gateway implementation, but if a simple tweak requires replacing the entire routeConfiguration of a listener, I feel like it defeats the whole separation of concerns idea that the Gateway API is supposed to address. It's basically like going back to the old idea of a statically configured httpd server.

I don't mean to be critical, I know it's still under heavy development and "use at your own risk" before the official march GA release, but my honest feedback is that I think for wide adoption / long term trust, people are going to need to be able to apply patches directly to security policies/routes. Ideally with some kind of applyAfter: <CRD Ref> and applyBefore: <CRD Ref> logic. In my case I managed to get everything 99% working very quickly out of the box which was awesome; However because of the 1% that didn't work -a single missing response header- that there doesn't appear to be any way to add after hours and hours of fruitless attempts -I'm having to switch out the entire routing framework in our cluster from EnvoyGateway to Contour.

apiVersion: gateway.envoyproxy.io/v1alpha1
kind: EnvoyPatchPolicy
metadata:
  name: {{.Values.api.name}}-api-cors
  labels:
    {{- include "project.labels" . | nindent 4 }}
    {{- include "project.api.labels" . | nindent 4 }}
spec:
  targetRef:
    group: gateway.networking.k8s.io
    kind: Gateway
    name: envoy-main
  type: JSONPatch
  jsonPatches:
    - type: type.googleapis.com/envoy.config.route.v3.RouteConfiguration
      name: project-dev-main/envoy-main/https
      operation:
        op: replace
        path: virtualHosts/0/routes/0/typedPerFilterConfig/envoy.filters.http.cors
        value:
          "@type": type.googleapis.com/envoy.extensions.filters.http.cors.v3.CorsPolicy
          allow_origin_string_match:
            - suffix: project.dev.example.io
          allowCredentials: true

I really like the idea of the envoy gateway precisely because, unlike all the other Gateway implementations, this one strives to be as low-level and customizable as possible. I was very frustrated with the AWS Gateway API implementation because not only does it not provide any mechanism for North-South traffic handling -it forces you to use an entirely new paid-for service that adds tons of complexity. In general the thing I find most frustrating about Kubernetes is that I often find that Operators are either too simplistic -or address enterprise grade problems that only Google scale companies really have to deal with. There is no middle ground.

In my case I basically just want to be able to set auth/cors for webApps I manage without having to create an ingress for every application service that needs to be exposed or install/maintain a complex service mesh that is overkill for my organization's needs.

Your combo of HttpRoute + SecurityPolicy directly addresses those needs -especially as it helps keep application resources confined/grouped as much as possible by namespace. I just wish it was easier to add fine-grained patches so I could feel confident I won't need to switch it out!

[apjoseph]

Turns out Contour has the same issue. I would have thought most people using JWTs are generating them via oauth2 and then passing them to an external API. Are people using JWTs without http-only cookies these days? How do you secure them?

[zhaohuabing]

@apjoseph Thanks for experimenting with the SecurityPolicy and reporting this issue.

I guess it's because the cookie in your environment is an "http-only" cookie, and the request is sent out through a script?

Would it help if allowCredentials is exposed to the CORS configuration of the SecurityPolicy?

[apjoseph]

@apjoseph Thanks for experimenting with the SecurityPolicy and reporting this issue.

I guess it's because the cookie in your environment is an "http-only" cookie, and the request is sent out through a script?

Would it help if allowCredentials is exposed to the CORS configuration of the SecurityPolicy?

@zhaohuabing Yes allowCredentials would definitely help!

To help describe the context of the issue, say you have two services behind a Gateway. One is web.example.com - a standard react app service and the other is api.example.com - a node js service which is meant to serve data to the web app.

If you add the oidc SecurityPolicy to the httproute for web.example.com and login using a standard pkce oauth2 flow, you'll be redirected back to /oauth2/callback with a code. Envoy handles that route directly and makes the external request to the configured oauth2 token endpoint and sends a redirect response to the last page and sets a BearerToken http-only cookie containing the token.

That part works great.

Where things break down is that in the JS for web.example.com you want to make a request to your api service at api.example.com. So something like GET /todos.

In order to use the http cookie, the browser is going to first make an OPTIONS request to /todos. It will expect the allow credentials header in the response. If that header isn't present in the OPTIONS request, then the browser will not allow the request to GET /todos.

Therefore you need to define two separate httproutes for the api.example.com service.

One to handle OPTIONS pre-flight requests with a matcher like

- prefix: /
  method: OPTIONS

You'd need to attach a SecurityPolicy to this HTTPRoute that adds the CORS filter.

Second, you need a HTTPRoute for all non-options methods with a matcher of

- prefix: / 

Which is secured by a SecurityPolicy that defines the JWT filter.