OIDC slows down application startup time in native mode

[igor-dmitriev]

Description

We are using Quarkus with GraalVM native-image which is deployed to AWS Lambda, we noticed significant performance loss when we added io.quarkus:quarkus-oidc dependency.
It is needed to have Auth0 integration.

implementation 'io.quarkus:quarkus-oidc'

quarkusPlatformVersion=1.12.2.Final

application.properties

quarkus.log.console.format=%d{yyyy-MM-dd HH:mm:ss} | %X{userId} [%-5p] %c{1} - %m%n
quarkus.log.category."software.amazon.awssdk".level=WARN
quarkus.log.category."software.amazon.awssdk.request".level=DEBUG
quarkus.log.category."io.quarkus.bootstrap".level=WARN
quarkus.log.category."com.igor.repository".level=DEBUG

quarkus.oidc.enabled=false

quarkus.oidc.auth-server-url=${AUTH_SERVER_URL}
quarkus.oidc.client-id=${CLIENT_ID}

quarkus.http.auth.permission.default.paths=/*
quarkus.http.auth.permission.default.policy=permit

quarkus.http.auth.permission.swagger.paths=/openapi,/documentation/*,/branding/*
quarkus.http.auth.permission.swagger.policy=permit

quarkus.swagger-ui.title=API
quarkus.swagger-ui.theme=outline
quarkus.swagger-ui.path=/documentation

quarkus.http.cors=true
quarkus.http.cors.origins=*
quarkus.http.cors.headers=accept,authorization,content-type,x-requested-with
quarkus.http.cors.methods=GET,POST,PUT,PATCH,DELETE,OPTIONS

quarkus.datasource.db-kind=postgresql
quarkus.datasource.username=${JDBC_LOGIN}
quarkus.datasource.password=${JDBC_PASSWORD}
quarkus.datasource.jdbc.url=${JDBC_URL}
quarkus.mybatis.map-underscore-to-camel-case=true

quarkus.native.additional-build-args=--report-unsupported-elements-at-runtime

This is how it works without oidc:

as you may see start-up time is 0.2 seconds

and this is how it works with oidc:

We tested it multiple times
+0.5 seconds in average

Implementation ideas

[quarkus-bot]

/cc @pedroigor, @sberyozkin

[sberyozkin]

@igor-dmitriev Can you please experiment with Quarkus 1.13.0.CR1 ? It has a new implementation

[sberyozkin]

@igor-dmitriev Also, if the Auth0 bearer token in this case is in JWT format then you can try:

quarkus.oidc.auth-server-url=${AUTH_SERVER_URL}
quarkus.oidc.client-id=${CLIENT_ID}
quarkus.oidc.discovery-enabled=false
quarkus.oidc.jwks-path=${relativet-auth0-jwks-path)

and if it is an opaque/binary token then:

quarkus.oidc.auth-server-url=${AUTH_SERVER_URL}
quarkus.oidc.client-id=${CLIENT_ID}
quarkus.oidc.discovery-enabled=false
quarkus.oidc.introspection-path=${relativet-auth0-introspection-path)

it will save on quarkus-oidc fetching the well-known config from Auth0 at the the start up time

[igor-dmitriev]

@sberyozkin thanks for the update, I will try it out

[igor-dmitriev]

@sberyozkin so, I tried it out,
it fails with an error on 1.13.0.CR1 version:

Execution failed for task ':quarkusGenerateCode'.
> Could not resolve all dependencies for configuration ':runtimeClasspath'.
   > Could not find io.quarkus:quarkus-universe-bom:1.13.0.CR1.
     Required by:
         project :
   > Could not find io.quarkus:quarkus-amazon-lambda-http-v1:.
     Required by:
         project :
   > Could not find net.java.dev.jna:jna:.
     Required by:
         project :
   > Could not find io.quarkus:quarkus-hibernate-validator:.
     Required by:
         project :
   > Could not find io.quarkus:quarkus-oidc:.
     Required by:
         project :
   > Could not find io.quarkus:quarkus-smallrye-openapi:.
     Required by:
         project :

* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.

regarding the discovery, it works a little bit better when discovery is disabled:

quarkus.oidc.auth-server-url=${AUTH_SERVER_URL}
quarkus.oidc.client-id=${CLIENT_ID}
quarkus.oidc.discovery-enabled=false
quarkus.oidc.jwks-path=.well-known/jwks.json

Results:

now it's 0.5 seconds in average

[sberyozkin]

@igor-dmitriev I believe for CR* releases one needs to use quarkus-bom, cc @gsmet, give it a try please

[sberyozkin]

@igor-dmitriev if Auth0 jwks have the embedded cert chains then the old implementation would decode and try to validate partially (only the root cert) - but the new one does not do it as it does not use them and because they have come from the trusted channel anyway - so here something might be saved as well.
Also, additionally, we can also look at postponing (but only optionally) the OIDC provider connection initialization until runtime as part of #15599 (the init cost would be at the first request time)

[igor-dmitriev]

@sberyozkin got it, thanks for the update, looking forward to it, having startup time ~= 200ms for Java-based application is fantastic.

[sberyozkin]

@igor-dmitriev I've been thinking about this issue - so there is no need to introduce a property to delay quarkus-oidc initialization until the very first request as one can already do it with a custom resolver.

Instead of

quarkus.oidc.auth-server-url=${AUTH_SERVER_URL}
quarkus.oidc.client-id=${CLIENT_ID}
quarkus.oidc.discovery-enabled=false
quarkus.oidc.jwks-path=.well-known/jwks.json

you can do

# will not be required in 1.14.0.Final
quarkus.oidc.tenant-enabled=false

and register a custom TenantContextResolver bean where you'd do:

// in post construct
OidcTenantConfig config = new OidcTenantConfig
config.setAuthServerUrl(System.geProperty("AUTH_SERVER_URL"};
// and also set these properties
//quarkus.oidc.client-id=
//quarkus.oidc.discovery-enabled=false
//quarkus.oidc.jwks-path=.well-known/jwks.json

Can you give it a try please.
OIDC code needs to fetch JWK keys and process them so it take a bit of time if done at the startup

[igor-dmitriev]

Hi @sberyozkin , it is not working, cause it is required to provide auth-server-url and client-id, otherwise it fails with an error:

Both '%sauth-server-url' and '%sclient-id' properties must be configured

when I added them and added a custom TenantContextResolver as well, it does not give any positive effect, unfortunately

@ApplicationScoped
public class CustomTenantConfigResolver implements TenantConfigResolver {

  OidcTenantConfig config;

  @PostConstruct
  public void init() {
    config = new OidcTenantConfig();
    config.setAuthServerUrl("BLABLA");
    config.setClientId("BLABLA");
    config.setJwksPath(".well-known/jwks.json");
    config.setDiscoveryEnabled(false);
    config.setIntrospectionPath(".well-known/openid-configuration");
    config.setTenantId("test");
  }

  @Override
  public OidcTenantConfig resolve(RoutingContext context) {
    return config;
  }
}

application.properties

quarkus.oidc.auth-server-url=BLABLA
quarkus.oidc.client-id=BLABLA
quarkus.oidc.tenant-enabled=false

[sberyozkin]

@igor-dmitriev

it is not working, cause it is required to provide auth-server-url and client-id, otherwise it fails with an error...

I only proto-typed the code, so indeed setting client-id was also expected.
OK, so you see it working in principle with TenantConfigResolver - thanks for trying it.

With this resolver, OidcRecorder does not set any code really which would do something at the start up.
The only other thing that I'd like to ask you to try is switch to 1.13.0.Final - it has a new OIDC channel support code - so perhaps you'd see some difference there.
Please keep this PR in mind, #16157, but Auth0 may not be affected by the problem that PR has addressed - but in case it is affected then 1.13.1.Final which is due next week will have it fixed

Please experiment further

[sberyozkin]

By the way quarkus-oidc, in its SNAPSHOT code, will start with auth-server-url in the dev mode (though it is not related to this issue)

[igor-dmitriev]

@sberyozkin yes, I tried it with 1.13.0.Final, nothing changed, I do not see the difference, application start-up time seems still affected, ~=0.5 seconds in AWS Lambda

[igor-dmitriev]

@sberyozkin I tried your proposal with 1.13.1.Final, it is working better than with 1.13.0.Final, now I can see that start-up time is ~0.3 second. So, auth stuff is deferred to the first call.