CDI context propagation improvements for the reactive stack

[mkouba]

This PR implements a new way to store the ArC request scope. Instead of storing that object in a ThreadLocal (which is problematic in an event loop-based architecture), the request scope is stored in the Vert.x duplicated context which is used for the whole reactive/async processing. If there is no duplicated context, it falls back to a thread-local.

This PR makes accessing request-scope beans a lot simpler in reactive applications. The request scope is available as soon as the continuations are executed on the same duplicated context. We aware that if this access happens after the termination of the scope, an error is reported (as it's clearly misuing the concepts).

ArC

Introduce the VertxCurrentContextFactory so that the Vert.x duplicated context can be used to store the "current context" for normal scopes. If we're not on the Vert.x duplicated context we fallback to the ThreadLocal (or FastThreadLocal respectively).

resteasy-reactive

Don't clear our CDI current request when suspending.

This was done to prevent subsequent requests running on the same thread as a suspended request from accessing the former's data. With the advent of DuplicatedContext-backed storage, there is no longer any chance of mixing data, so there is no need to clear it out.

Furthermore, by not clearing out current requests, the code that accesses the request scoped CurrentVertxRequest
that is executed while the request is suspended can now work even if context propagation is not in play.

Tests

Add leak detection tests for resteasy reactive, graphql, and reactive rest client. Also, improve the OpenTelemetry reactive tests.

OpenTelemetry

Only register the OpenTelemetryClientFilter for RESTEasy Client Classic. Use Capabilities to determine when to register the OpenTelemetryClientFilter.

[mkouba]

@Sanne would you be able to benchmark this PR (with CP disabled) to get some actual numbers?

[mkouba]

@radcortez @brunobat Hey guys, the OpenTelemetryReactiveTest#multiple() test fails in this WIP because it expects 7 spans but it only gets 6. Unfortunately, the test does not contain any description and my OTel tracing knowledge is minimal. Would you care to take a look? Any details would be appreciated...

@cescoffier may also add some relevant info because he fixed some OTel reactive problem recently.

[cescoffier]

I didn't fix anything; I've found some code that would need to be changed to support this PR. OTel is duplicating duplicated context to create a new span. With this PR, you cannot do that anymore. We need to find another way, like marking the context or something like this.

[mkouba]

I didn't fix anything

Eh, yes that commit is actually pretty old :D

[geoand]

Also, please rebase onto main in order to pick up a CI fix.

[brunobat]

@radcortez @brunobat Hey guys, the OpenTelemetryReactiveTest#multiple() test fails in this WIP because it expects 7 spans but it only gets 6. Unfortunately, the test does not contain any description and my OTel tracing knowledge is minimal. Would you care to take a look? Any details would be appreciated...

@cescoffier may also add some relevant info because he fixed some OTel reactive problem recently.

I believe we are missing a span like this, for the 2nd client call, with QueryParam Goku:
SdkSpan{traceId=f6bccae8b9c40899056cfb2236529f82, spanId=2d51ec72a1e13750, parentSpanContext=ImmutableSpanContext{traceId=f6bccae8b9c40899056cfb2236529f82, spanId=f67817406f93a6ac, traceFlags=01, traceState=ArrayBasedTraceState{entries=[]}, remote=false, valid=true}, name=HTTP GET, kind=CLIENT, attributes=AttributesMap{data={http.method=GET, http.url=http://localhost:8081/reactive?name=Naruto, http.user_agent=Resteasy Reactive Client, http.status_code=200, http.response_content_length=12}, capacity=128, totalAddedValues=5}, status=ImmutableStatusData{statusCode=UNSET, description=}, totalRecordedEvents=0, totalRecordedLinks=0, startEpochNanos=1661524694185603792, endEpochNanos=1661524696249212208}

[cescoffier]

@brunobat yes, because the initial code was switching to another duplicated context without going back to it after the operation.

So either we need to implement this second switch or we find another way to trigger the creation of the span.

[mkouba]

@cescoffier ^ problem with formatting...

[cescoffier]

@mkouba fixed. I added the RR and a GrapQL tests as discussed.
I'm now looking at the OTel issue.

[radcortez]

I've changed the registration of the OpenTelemetryClientFilter only to be applied in the RESTEasy Classic Client. This avoids the double spans for the Reactive Client, which also uses the Vert.x Tracer.

[radcortez]

I would also like to add some additional tests on the OTel side.

[cescoffier]

@radcortez what about testing that the span "tree" is correct (what I did manually yesterday)?

The purpose would be to detect changes in the structure when we add more OTel support in Mutiny (because typically, for mutiny, the tree is slightly different with the last span having two parents).

[quarkus-bot]

/cc @mjurc, @rsvoboda

Addressed!

[rsvoboda]

Triggered adhoc GitHub Action Daily run to check possible side-effects and the run was green.
No regression was detected with our TS (https://github.com/quarkus-qe/quarkus-test-suite).