OIDC 1.10.0.CR1 - event loop thread blocked

[missourian55]

Describe the bug
Upgraded an existing app from 1.9.2.Final to 1.10.0.CR and I am getting below error

2020-11-12 01:14:14,178 INFO  [io.quarkus] (main) Installed features: [amazon-s3, cache, cdi, kubernetes, logging-gelf, mutiny, oidc, rest-client, resteasy, resteasy-jsonb, resteasy-mutiny, security, smallrye-context-propagation]
2020-11-12 01:17:13,262 ERROR [io.qua.ver.htt.run.QuarkusErrorHandler] (vert.x-eventloop-thread-0) HTTP Request to /s3/s3.html?state=ea63119a-310d-4c65-9c16-5aded25683ec&session_state=9616cbb2-8869-42e1-ad81-1964a72ee7ff&code=69ad933f-9481-45f6-b731-28960124e1f5.9616cbb2-8869-42e1-ad81-1964a72ee7ff.91b23959-4d87-4fbe-a790-69b838fbe4a0 failed, error id: 479eb3ed-e9db-49e0-a817-28a3d16c0eb0-1: 
java.lang.IllegalStateException: The current thread cannot be blocked: vert.x-eventloop-thread-0
	at io.smallrye.mutiny.operators.UniBlockingAwait.await(UniBlockingAwait.java:29)
	at io.smallrye.mutiny.groups.UniAwait.atMost(UniAwait.java:61)
	at io.smallrye.mutiny.groups.UniAwait.indefinitely(UniAwait.java:42)
	at io.quarkus.oidc.runtime.OidcIdentityProvider.verifyCodeFlowAccessToken(OidcIdentityProvider.java:243)
	at io.quarkus.oidc.runtime.OidcIdentityProvider.validateTokenWithOidcServer(OidcIdentityProvider.java:96)
	at io.quarkus.oidc.runtime.OidcIdentityProvider.authenticate(OidcIdentityProvider.java:84)
	at io.quarkus.oidc.runtime.OidcIdentityProvider.access$100(OidcIdentityProvider.java:37)
	at io.quarkus.oidc.runtime.OidcIdentityProvider$1$1.get(OidcIdentityProvider.java:71)
	at io.quarkus.oidc.runtime.OidcIdentityProvider$1$1.get(OidcIdentityProvider.java:59)
	at io.smallrye.mutiny.operators.UniCreateFromDeferredSupplier.subscribing(UniCreateFromDeferredSupplier.java:24)
	at io.smallrye.mutiny.operators.UniSerializedSubscriber.subscribe(UniSerializedSubscriber.java:54)
	at io.smallrye.mutiny.operators.UniSerializedSubscriber.subscribe(UniSerializedSubscriber.java:49)
	at io.smallrye.mutiny.operators.AbstractUni.subscribe(AbstractUni.java:30)
	at io.smallrye.mutiny.context.ContextPropagationUniInterceptor$2.lambda$subscribing$0(ContextPropagationUniInterceptor.java:48)
	at io.smallrye.context.SmallRyeThreadContext.lambda$withContext$1(SmallRyeThreadContext.java:530)
	at io.smallrye.mutiny.context.ContextPropagationUniInterceptor$2.subscribing(ContextPropagationUniInterceptor.java:48)
	at io.smallrye.mutiny.operators.UniSerializedSubscriber.subscribe(UniSerializedSubscriber.java:54)
	at io.smallrye.mutiny.operators.UniSerializedSubscriber.subscribe(UniSerializedSubscriber.java:49)
	at io.smallrye.mutiny.operators.AbstractUni.subscribe(AbstractUni.java:30)
	at io.smallrye.mutiny.operators.UniOnItemTransformToUni.handleInnerSubscription(UniOnItemTransformToUni.java:57)
	at io.smallrye.mutiny.operators.UniOnItemTransformToUni.invokeAndSubstitute(UniOnItemTransformToUni.java:43)
	at io.smallrye.mutiny.operators.UniOnItemTransformToUni$2.onItem(UniOnItemTransformToUni.java:74)
	at io.smallrye.mutiny.context.ContextPropagationUniInterceptor$1.lambda$onItem$1(ContextPropagationUniInterceptor.java:32)
	at io.smallrye.context.SmallRyeThreadContext.lambda$withContext$1(SmallRyeThreadContext.java:530)
	at io.smallrye.mutiny.context.ContextPropagationUniInterceptor$1.onItem(ContextPropagationUniInterceptor.java:32)
	at io.smallrye.mutiny.operators.UniSerializedSubscriber.onItem(UniSerializedSubscriber.java:86)
	at io.smallrye.mutiny.context.ContextPropagationUniInterceptor$1.lambda$onItem$1(ContextPropagationUniInterceptor.java:32)
	at io.smallrye.context.SmallRyeThreadContext.lambda$withContext$1(SmallRyeThreadContext.java:530)
	at io.smallrye.mutiny.context.ContextPropagationUniInterceptor$1.onItem(ContextPropagationUniInterceptor.java:32)
	at io.smallrye.mutiny.operators.UniSerializedSubscriber.onItem(UniSerializedSubscriber.java:86)

Expected behavior
OIDC hybrid auth should work

Actual behavior
Getting the above stacktrace

Configuration

quarkus.oidc.auth-server-url=http://keycloak.com
quarkus.oidc.client-id=backend
quarkus.oidc.credentials.secret=71234b
quarkus.oidc.application-type=hybrid
quarkus.oidc.roles.source=accesstoken
quarkus.oidc.roles.role-claim-path=scope
quarkus.http.auth.permission.authenticated.paths=/*
quarkus.http.auth.permission.authenticated.policy=authenticated

Environment (please complete the following information):

• Output of uname -a or ver: mac os & Linux
• Output of java -version: Java 11
• GraalVM version (if different from Java): NA
• Quarkus version or git rev: 1.10.0.CR1
• Build tool (ie. output of mvnw --version or gradlew --version): mvnw

Additional context
@sberyozkin Let me know if you need any additional details

[ghost]

/cc @Ladicek, @cescoffier, @geoand, @jmartisk, @jponge, @kenfinnigan, @manovotn, @matejvasek, @mkouba, @patriot1burke, @pedroigor, @phillip-kruger, @radcortez, @sberyozkin

[cescoffier]

@sberyozkin We need to eliminate all the "await". It's working by accident as the Uni items are cached, but it's not a good pattern to use these methods. It may end up blocking the event loop indefinitely if the retrieval hangs.

CC @jponge

[sberyozkin]

@cescoffier yeah, I'm very sure the last enhancement is not at fault, it is
io.quarkus.oidc.runtime.OidcIdentityProvider.verifyCodeFlowAccessToken(OidcIdentityProvider.java:243) which is broken (which was not touched at all in 1.10.0.CR1) in the way it uses Uni, I've been planning to address it as part of #8559. If anything, the last PR has improved the flow and exposed this bug :-)

@missourian55 If you are using Keycloak then you don't need quarkus.oidc.roles.source=accesstoken I think, can you drop it and check if it works ? Or if you can configure Keycloak to support MP JWT it should also do

[sberyozkin]

Hi @gsmet, I think quarkus-bot is a bit too active here :-), arc is not involved, smallrye- is not, I guess, in this particular case, since we have OIDC and event loop blocked, it would be cool if it would CC to Clement, Julien, myself, Pedro and Stuart; for smallrye-jwt - myself and Roberto, also may be @gastaldi and @FroMage for the security issues as well if they don't mind ?

[gsmet]

@sberyozkin yeah, so it reacts exactly as the old GH action. The issue here is that we have the list of installed extensions which triggered a lot of rules. I'm not sure if we want to limit it to the title or not.

[gsmet]

@cescoffier @sberyozkin I consider this one blocking for the release so please make sure it is fixed soon.

[sberyozkin]

@gsmet Yeah, I'm on it right now; FYI, it has been a dormant issue; I have a very quick fix which I'll open later today if I get stuck with a proper Uni fix