Seamlessly Integrate Quarkus/smallrye-jwt with AWS Application Load Balancer (ALB)

[AdamBien]

Description

The AWS Application Load Balancer (ALB) can authenticate users through an identity provider (IdP) that is OpenID Connect (OIDC) compliant (https://docs.aws.amazon.com/elasticloadbalancing/latest/application/listener-authenticate-users.html).

However, the AWS Application Load Balancer (ALB) creates invalid JWTs and requires dynamic public key lookups from the kid header. See documentation: (https://docs.aws.amazon.com/elasticloadbalancing/latest/application/listener-authenticate-users.html#user-claims-encoding):

"...
Considerations
Standard libraries are not compatible with the padding that is included in the Application Load Balancer authentication token in JWT format.
..."

Token example:

eyJ0eXAiOiJKV1QiLCJraWQiOiJjMmY4MGM4Yi1jMDVjLTQwNjgtYWYxNC0xNzI5OWY3ODk2YjEiLCJhbGciOiJFUzI1NiIsImlzcyI6Imh0dHBzOi8vY29nbml0by1pZHAuZXUtY2VudHJhbC0xLmFtYXpvbmF3cy5jb20vZXUtY2VudHJhbC0xX015UnJPQ0hRdyIsImNsaWVudCI6IjRmbXZodDIydGpyZ2Q3ZDNrM3RnaHR0Y3Q3Iiwic2lnbmVyIjoiYXJuOmF3czplbGFzdGljbG9hZGJhbGFuY2luZzpldS1jZW50cmFsLTE6MTk3MjgwOTU4MjI1OmxvYWRiYWxhbmNlci9hcHAvZWNzLXdpdGgtY29nbml0by1sYi82Mjg0YmU2NWI4MjdjNTk4IiwiZXhwIjoxNjg3NzQ4MDQ1fQ==.eyJzdWIiOiIyM2Q0OThiMi0zMDMxLTcwZDItOGExNS00OWRkODg2YTA4N2IiLCJlbWFpbF92ZXJpZmllZCI6InRydWUiLCJlbWFpbCI6ImR1a2VAc3VuLmNvbSIsInVzZXJuYW1lIjoiZHVrZSIsImV4cCI6MTY4Nzc0ODA0NSwiaXNzIjoiaHR0cHM6Ly9jb2duaXRvLWlkcC5ldS1jZW50cmFsLTEuYW1hem9uYXdzLmNvbS9ldS1jZW50cmFsLTFfTXlSck9DSFF3In0=.Jd7RXHsOj8vw2b4irZCxxWO-0UQBZ2X1bRNsKZ9D02JWJaNOvOnrV8T-qrcmWNpl7MjNhsGSm1C4e2rAjaF0jg==

Problem:

Each section ends with an unnecessary padding character, which yields the JWT invalid.

Warnings from jwt.io:

Your JWT signature is not encoded correctly using base64url (https://tools.ietf.org/html/rfc4648#section-5). Note that padding ("=") must be omitted as per https://tools.ietf.org/html/rfc7515#section-2

Your JWT header is not encoded correctly using base64url (https://tools.ietf.org/html/rfc4648#section-5). Note that padding ("=") must be omitted as per https://tools.ietf.org/html/rfc7515#section-2

Your JWT payload is not encoded correctly using base64url (https://tools.ietf.org/html/rfc4648#section-5). Note that padding ("=") must be omitted as per https://tools.ietf.org/html/rfc7515#section-2

Implementation ideas

To make smallrye-jwt work with alb, the following issues have to be resolved:

1. (minor) The public key has to be looked up dynamically and fetched from the ALB. Quarkus configuration is static and set at build time.
   mp.jwt.verify.publickey.location=https://public-keys.auth.elb.eu-central-1.amazonaws.com/[fetch from kid header]
   The "kid" resides in the first JWT section (header):
   { "typ": "JWT", "kid": "XXXX-a34d-4255-a779-YYYYYYY", ... }
   The key does not change frequently and can be "hardcoded" for evaluation purposes.

2. (primary) The format of the JWT token issued by the ALB is invalid:
   from AWS / ALB documentation: The JWT Signature and the JWT Header contain a padding character "=" which is not allowed by the spec.
   Possible solution:

3. Parse JWT header

4. Obtain the kid value

5. Use the value to fetch (and cache) the public key with Java HTTP client: GET https://public-keys.auth.elb.eu-central-1.amazonaws.com/[kid value]

6. Verify the signature with Java built-in functionality

7. Remove the padding ("=") characters

8. Use e.g. Smallrye JWT to parse the token and use the principal/claims.

The extension of the MP / Smallrye JWT (Quarkus) mechanism (quarkus-smallrye-jwt extension) itself would seamlessly integrate ECS (Fargate), EKS, AWS Lambda applications running on quarkus with ALB OIDC functionality. This direct integration increases security, developer experience, and maintainability.

[quarkus-bot]

/cc @Ladicek (smallrye), @jmartisk (smallrye), @phillip-kruger (smallrye), @radcortez (smallrye), @sberyozkin (jwt)

[sberyozkin]

Hi @AdamBien Thanks for opening this issue.

I'm not sure right now what is the best way to handle it because JWT signature verification is all about following the JOSE JWS signature verification process - where base64url encoding is essential, in case of ES256 the example is here:
https://www.rfc-editor.org/rfc/rfc7515#appendix-A.3

and I'd rather a dedicated JOSE library (jose4j in Quarkus's case) handle it - I did that code myself awhile back in another project, indeed, using Java primitives, but now I'd prefer jose4j do it which has some fine checks about correct elliptic curves etc.

I'll need to have a look if we can ask Jose4j to verify the already decoded headers/payload, the solution I have in mind is to have a custom AWS factory:

https://quarkus.io/guides/security-jwt#custom-factories

and ship a dedicated smallrye-jwt-aws module with this factory. I'll experiment and see what we can do about it

Cheers

[AdamBien]

A good idea! A dedicated extension would be even more convenient, and we could rely more on conventions and streamline the configuration.

[sberyozkin]

@AdamBien For the moment I was thinking of having an extra module or some kind of customization support at the smallrye-jwt level, I haven't thought yet about a dedicated Quarkus extension (in Quarkiverse). Supporting a custom key resolver would likely make sense as there are cases like Google Firebase etc.

Can I clarify something before we try to do something about this case, can it be the case such a token has already been verified at the AWS ALB level, before it reaches Quarkus ?

[sberyozkin]

@AdamBien I have a unit test where this token is successfully verified:

smallrye/smallrye-jwt#707

(so the only outstanding issue is a dynamic key resolution, but we might not even require users register a custom key resolver, but simply add a dedicated key format property like AWS_ALB_PEM or something like that to give smallryw-jwt a hint that the the configured key location has to be concatenated with kid in order to fetch it)

Jose4j is not worried about base64url vs base64 because, if we think about it, the only idea of preferring the former is to have the encoded sequence URL friendly, padding = characters can get URL encoded by the browsers.

This is somewhat unexpected though: @b---c, Hi, can you please clarify, I think it is not a problem that jose4j can consume JWS sequences where parts are Base64 encoded as the padding characters are not affecting the signature verification.

Thanks

[AdamBien]

@AdamBien For the moment I was thinking of having an extra module or some kind of customization support at the smallrye-jwt level, I haven't thought yet about a dedicated Quarkus extension (in Quarkiverse). Supporting a custom key resolver would likely make sense as there are cases like Google Firebase etc.

Can I clarify something before we try to do something about this case, can it be the case such a token has already been verified at the AWS ALB level, before it reaches Quarkus ?

The ALB documentation notes (https://docs.aws.amazon.com/elasticloadbalancing/latest/application/listener-authenticate-users.html):

After your load balancer authenticates a user successfully, it sends the user claims received from the IdP to the target. The load balancer signs the user claim so that applications can verify the signature and verify that the claims were sent by the load balancer.

The load balancer adds the following HTTP headers:

x-amzn-oidc-accesstoken
The access token from the token endpoint, in plain text.
x-amzn-oidc-identity
The subject field (sub) from the user info endpoint, in plain text.
x-amzn-oidc-data
The user claims, in JSON web tokens (JWT) format.

[AdamBien]

@AdamBien I have a unit test where this token is successfully verified:

smallrye/smallrye-jwt#707

(so the only outstanding issue is a dynamic key resolution, but we might not even require users register a custom key resolver, but simply add a dedicated key format property like AWS_ALB_PEM or something like that to give smallryw-jwt a hint that the the configured key location has to be concatenated with kid in order to fetch it)

You mean: "the configured key location has to be concatenated resolved with kid key in order to fetch it"?

Jose4j is not worried about base64url vs base64 because, if we think about it, the only idea of preferring the former is to have the encoded sequence URL friendly, padding = characters can get URL encoded by the browsers.

This is somewhat unexpected though: @b---c, Hi, can you please clarify, I think it is not a problem that jose4j can consume JWS sequences where parts are Base64 encoded as the padding characters are not affecting the signature verification.

Thanks

[AdamBien]

A minimal AWS configuration in the stock smallrye-jwt is even better. A "core" extension is more trustworthy and easier to maintain.

[sberyozkin]

:@b---c, sorry, the padding characters in headers and the body are verified, it is only in the signature part they make no difference, which is totally fine since it is just an envelope for the signature part, somehow I got confused, thanks

[sberyozkin]

@AdamBien FYI, smallrye/smallrye-jwt#707.

As far as the padding characters are concerned, all is good, here is copy of what I've typed there:

Note the token parts have been Base64 encoded, as opposed to Base64Url encoded, thus extra padding characters are added to the header and body and signature parts.

As far as the the header and body integrity verification is concerned, it makes no difference because the padding characters are included in the signing input - while for the signature it does not matter if its encoded form has extra padding characters or not - since the decoded signature is compared against the recalculated signature with the headers and body parts included in the signing input.

Base64-only encoded tokens introduce an interoperability problem - i.e not all libraries can consume such tokens - but it is not a problem of Jose4j - rather it is a problem of AWS ALB whose tokens may not be verified by all JOSE libraries (example, the warning from JWT.io).

Such tokens are also not URI friendly, as = may get URL encoded.

(CC @b---c - please correct if something above is not precise)

Thanks

[AdamBien]

@sberyozkin I just wanted capture your idea:

we can just have
smallrye.jwt.key-provider=AWS_ALB and internally a Key resolver replacing the default one will be created - a few well known non-JWK resolvers can be provided.

From the user perspective, the property above would introduce a "switch," enabling the dynamic key resolution mechanism and, if necessary, a special treatment for the non-standard JWT encoding.

This would significantly increase the user's (developer's) experience.

[sberyozkin]

@AdamBien Thanks, indeed, it looks it can be much simpler to let users set a property and have it working, as opposed to add another module bringing custom verification key resolution logic.
FYI, I've moved the PR to the draft state again, I forgot it needs some basic cache support, as having to fetch and build the key on every request would be a problem

[sberyozkin]

@AdamBien #34564 will resolve this issue, I'll need to do some minor doc updates in another PR. Refactoring smallrye-jwt to make it reactive would be a longer term goal, will clarify in the docs, since using the worker threads might be necessary

[sberyozkin]

Fixed by #34564, see also #34575.