Add JSON streaming for RESTEasy Reactive Jsonb and Jackson #20908

Sgitario · 2021-10-20T13:52:22Z

We now support json streaming. Example:

    @Path("stream-json/multi")
    @GET
    @Produces(SseMediaType.APPLICATION_STREAM_JSON)
    @RestSseElementType(MediaType.APPLICATION_JSON)
    public Multi<Message> multiStreamJson() {
        return Multi.createFrom().items(new Message("hello"), new Message("stef"));
    }

We support "application/x-ndjson" and "application/stream+json".

Moreover, I've added a new utility class called SseMediaType to be used by users. The implementation is similar to MediaType.

Questions:

Added for both JSONB and Jackson extensions, is this correct?
I've copied the existing tests we had for sse in JSONB to Jackson, is this correct?

Fix #13663

geoand · 2021-10-20T13:53:07Z

Glad to see this, thanks!

@FroMage is the expert on streaming, so let's get a review from him :)

Sgitario · 2021-10-20T13:53:34Z

...time/src/main/java/org/jboss/resteasy/reactive/server/handlers/PublisherResponseHandler.java

@@ -197,24 +242,40 @@ public void handle(ResteasyReactiveRequestContext requestContext) throws Excepti
            // media type negotiation and fixed entity writer set up, perhaps it's better than
            // cancelling the normal route?
            // or make this SSE produce build-time
-            MediaType[] mediaTypes = requestContext.getTarget().getProduces().getSortedMediaTypes();
-            if (mediaTypes.length != 1)
+            MediaType[] mediaTypes = requestContext.getTarget().getProduces().getSortedOriginalMediaTypes();


I had to use getSortedOriginalMediaTypes because when using application/stream+json, we were only getting application/json using the method getSortedMediaTypes.

quarkus-bot · 2021-10-20T14:04:59Z

This workflow status is outdated as a new workflow run has been triggered.

Failing Jobs - Building `1e32c23`

Status	Name	Step	Failures	Logs	Raw logs
✖	Initial JDK 11 Build	`Build`	Failures	Logs	Raw logs

Failures

⚙️ Initial JDK 11 Build #

- Failing: independent-projects/resteasy-reactive/server/runtime 
! Skipped: devtools/bom-descriptor-json docs extensions/agroal/deployment and 210 more

📦 independent-projects/resteasy-reactive/server/runtime

✖ Failed to execute goal net.revelc.code:impsort-maven-plugin:1.6.2:check (check-imports) on project resteasy-reactive: Imports are not sorted in /home/runner/work/quarkus/quarkus/independent-projects/resteasy-reactive/server/runtime/src/main/java/org/jboss/resteasy/reactive/server/handlers/PublisherResponseHandler.java

geoand · 2021-10-20T14:12:04Z

PR needs formatting

Sgitario · 2021-10-20T17:07:43Z

PR needs formatting

It should be fixed now

Sgitario · 2021-11-05T09:45:58Z

PR updated to resolve some new conflicts with main.

quarkus-bot · 2021-11-05T12:16:32Z

This workflow status is outdated as a new workflow run has been triggered.

Failing Jobs - Building `eb1994f`

Status	Name	Step	Failures	Logs	Raw logs
✖	Native Tests - Windows - hibernate-validator	`Build`	Failures	Logs	Raw logs

Failures

⚙️ Native Tests - Windows - hibernate-validator #

- Failing: integration-tests/hibernate-validator

📦 integration-tests/hibernate-validator

✖ Failed to execute goal io.quarkus:quarkus-maven-plugin:999-SNAPSHOT:build (default) on project quarkus-integration-test-hibernate-validator: Failed to build quarkus application

geoand · 2021-11-29T09:32:11Z

This now needs a rebase

Sgitario · 2021-11-29T11:36:29Z

This now needs a rebase

Done

geoand · 2021-11-29T11:37:28Z

@FroMage can you take a look at this one?

gsmet · 2022-01-12T16:33:33Z

@geoand what should we do with this one?

geoand · 2022-01-12T16:34:27Z

Ping @FroMage again 😊

FroMage

This looks good, except a few comments, and requires a mention in the documentation, linking to the stream specs.

...ctive/common/runtime/src/main/java/org/jboss/resteasy/reactive/common/util/SseMediaType.java

...ment/src/test/java/io/quarkus/resteasy/reactive/jackson/deployment/test/sse/SseResource.java

FroMage · 2022-01-18T13:38:16Z

Really sorry about the late review, this completely slipped past me.

Sgitario · 2022-01-19T14:47:38Z

@FroMage PR updated with one of your suggestions

FroMage

Looks great, just the question of the supertype of RestMediaType left.

...tive/common/runtime/src/main/java/org/jboss/resteasy/reactive/common/util/RestMediaType.java

quarkus-bot · 2022-01-20T09:47:52Z

This workflow status is outdated as a new workflow run has been triggered.

Failing Jobs - Building `0c102bf`

Status	Name	Step	Failures	Logs	Raw logs
✖	Gradle Tests - JDK 11 Windows	`Build`	Failures	Logs	Raw logs

Full information is available in the Build summary check run.

Failures

⚙️ Gradle Tests - JDK 11 Windows #

- Failing: integration-tests/gradle

📦 integration-tests/gradle

✖ io.quarkus.gradle.devmode.MultiModuleKotlinProjectDevModeTest.main line 22 - More details - Source on GitHub

org.awaitility.core.ConditionTimeoutException: Condition with lambda expression in io.quarkus.test.devmode.util.DevModeTestUtils that uses java.util.function.Supplier, java.util.function.Supplierjava.util.concurrent.atomic.AtomicReference was not fulfilled within 3 minutes.
	at org.awaitility.core.ConditionAwaiter.await(ConditionAwaiter.java:164)
	at org.awaitility.core.CallableCondition.await(CallableCondition.java:78)

quarkus-bot · 2022-01-20T11:06:31Z

This workflow status is outdated as a new workflow run has been triggered.

Failing Jobs - Building `f154712`

Status	Name	Step	Failures	Logs	Raw logs
✖	Initial JDK 11 Build	`Build`	Failures	Logs	Raw logs

Failures

⚙️ Initial JDK 11 Build #

- Failing: extensions/resteasy-reactive/quarkus-resteasy-reactive/deployment 
! Skipped: docs extensions/apicurio-registry-avro/deployment extensions/avro/deployment and 77 more

📦 extensions/resteasy-reactive/quarkus-resteasy-reactive/deployment

✖ Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.8.1:compile (default-compile) on project quarkus-resteasy-reactive-deployment: Compilation failure /home/runner/work/quarkus/quarkus/extensions/resteasy-reactive/quarkus-resteasy-reactive/deployment/src/main/java/io/quarkus/resteasy/reactive/server/deployment/QuarkusServerEndpointIndexer.java:[185,95] cannot find symbol symbol: method getSseElementType() location: variable method of type org.jboss.resteasy.reactive.server.model.ServerResourceMethod

quarkus-bot · 2022-01-20T13:45:07Z

This workflow status is outdated as a new workflow run has been triggered.

Failing Jobs - Building `9f1823d`

Status	Name	Step	Failures	Logs	Raw logs
✖	Gradle Tests - JDK 11 Windows	`Build`	Failures	Logs	Raw logs

Full information is available in the Build summary check run.

Failures

⚙️ Gradle Tests - JDK 11 Windows #

- Failing: integration-tests/gradle

📦 integration-tests/gradle

✖ io.quarkus.gradle.devmode.MultiModuleKotlinProjectDevModeTest.main line 22 - More details - Source on GitHub

org.awaitility.core.ConditionTimeoutException: Condition with lambda expression in io.quarkus.test.devmode.util.DevModeTestUtils that uses java.util.function.Supplier, java.util.function.Supplierjava.util.concurrent.atomic.AtomicReference was not fulfilled within 3 minutes.
	at org.awaitility.core.ConditionAwaiter.await(ConditionAwaiter.java:164)
	at org.awaitility.core.CallableCondition.await(CallableCondition.java:78)

✖ io.quarkus.gradle.devmode.QuarkusDevDependencyDevModeTest.main line 14 - More details - Source on GitHub

org.awaitility.core.ConditionTimeoutException: Condition with lambda expression in io.quarkus.test.devmode.util.DevModeTestUtils that uses java.util.function.Supplier, java.util.function.Supplierjava.util.concurrent.atomic.AtomicReference, java.util.concurrent.atomic.AtomicReferencejava.lang.String, java.lang.Stringboolean was not fulfilled within 1 minutes.
	at org.awaitility.core.ConditionAwaiter.await(ConditionAwaiter.java:164)
	at org.awaitility.core.CallableCondition.await(CallableCondition.java:78)

@get

We now support json streaming. Example: ``` @path("stream-json/multi") @get @produces(SseMediaType.APPLICATION_STREAM_JSON) @RestSseElementType(MediaType.APPLICATION_JSON) public Multi<Message> multiStreamJson() { return Multi.createFrom().items(new Message("hello"), new Message("stef")); } ``` We support "application/x-ndjson" and "application/stream+json". Moreover, I've added a new utility class called `SseMediaType` to be used by users. The implementation is similar to `MediaType`.

quarkus-bot · 2022-01-21T08:26:33Z

Failing Jobs - Building `d684ebe`

Status	Name	Step	Failures	Logs	Raw logs
✖	Gradle Tests - JDK 11 Windows	`Build`	Failures	Logs	Raw logs

Full information is available in the Build summary check run.

Failures

⚙️ Gradle Tests - JDK 11 Windows #

- Failing: integration-tests/gradle

📦 integration-tests/gradle

✖ io.quarkus.gradle.devmode.InjectQuarkusAppPropertiesDevModeTest.main line 19 - More details - Source on GitHub

org.awaitility.core.ConditionTimeoutException: Condition with lambda expression in io.quarkus.test.devmode.util.DevModeTestUtils that uses java.util.function.Supplier, java.util.function.Supplierjava.util.concurrent.atomic.AtomicReference, java.util.concurrent.atomic.AtomicReferencejava.lang.String, java.lang.Stringboolean was not fulfilled within 1 minutes.
	at org.awaitility.core.ConditionAwaiter.await(ConditionAwaiter.java:164)
	at org.awaitility.core.CallableCondition.await(CallableCondition.java:78)

FroMage

Looks great, thanks!

FroMage · 2022-01-21T16:23:37Z

CI failure unrelated: merging

We're wrongly using `/n` instead of `\n`. This was done in quarkusio#20908

quarkusio/quarkus#20908 quarkusio/quarkus#13663

quarkus-bot bot added the area/rest label Oct 20, 2021

geoand requested a review from FroMage October 20, 2021 13:53

Sgitario commented Oct 20, 2021

View reviewed changes

Sgitario force-pushed the add_stream_json branch from 1e32c23 to 482e38e Compare October 20, 2021 17:07

Sgitario force-pushed the add_stream_json branch from 482e38e to eb1994f Compare November 5, 2021 09:45

Sgitario force-pushed the add_stream_json branch from eb1994f to 2ba9c70 Compare November 29, 2021 11:36

FroMage requested changes Jan 18, 2022

View reviewed changes

...ctive/common/runtime/src/main/java/org/jboss/resteasy/reactive/common/util/SseMediaType.java Outdated Show resolved Hide resolved

...ment/src/test/java/io/quarkus/resteasy/reactive/jackson/deployment/test/sse/SseResource.java Outdated Show resolved Hide resolved

Sgitario force-pushed the add_stream_json branch from 2ba9c70 to 43a5a92 Compare January 19, 2022 14:47

Sgitario force-pushed the add_stream_json branch from 43a5a92 to 0c102bf Compare January 20, 2022 07:29

quarkus-bot bot added the area/documentation label Jan 20, 2022

Sgitario requested review from FroMage and geoand January 20, 2022 07:30

FroMage requested changes Jan 20, 2022

View reviewed changes

...tive/common/runtime/src/main/java/org/jboss/resteasy/reactive/common/util/RestMediaType.java Outdated Show resolved Hide resolved

Sgitario force-pushed the add_stream_json branch from 0c102bf to f154712 Compare January 20, 2022 10:42

Sgitario requested a review from FroMage January 20, 2022 10:42

Sgitario force-pushed the add_stream_json branch from f154712 to 9f1823d Compare January 20, 2022 11:08

Sgitario added 2 commits January 21, 2022 07:02

Deprecate @RestSseElementType in favour of @RestStreamElementType

d684ebe

Sgitario force-pushed the add_stream_json branch from 9f1823d to d684ebe Compare January 21, 2022 06:02

FroMage approved these changes Jan 21, 2022

View reviewed changes

FroMage merged commit 6ed8eb2 into quarkusio:main Jan 21, 2022

quarkus-bot bot added this to the 2.8 - main milestone Jan 21, 2022

Sgitario deleted the add_stream_json branch January 22, 2022 07:11

Sgitario added a commit to Sgitario/quarkus that referenced this pull request Mar 9, 2022

Resteasy Reactive: Fix line separator for "x-ndjson" and "stream+json".

a619566

We're wrongly using `/n` instead of `\n`. This was done in quarkusio#20908

Sgitario mentioned this pull request Mar 9, 2022

In Resteasy Reactive, fix line separator for "x-ndjson" and "stream+json". #24200

Merged

manusa added a commit to manusa/yakd that referenced this pull request Jun 7, 2023

refactor: replace RestSseElementType with RestStreamElementType

b147bb1

quarkusio/quarkus#20908 quarkusio/quarkus#13663

Add JSON streaming for RESTEasy Reactive Jsonb and Jackson #20908

Add JSON streaming for RESTEasy Reactive Jsonb and Jackson #20908

Conversation

Sgitario commented Oct 20, 2021 • edited by geoand Loading

geoand commented Oct 20, 2021 • edited Loading

Sgitario Oct 20, 2021

Choose a reason for hiding this comment

quarkus-bot bot commented Oct 20, 2021 • edited Loading

Failing Jobs - Building 1e32c23

Failures

⚙️ Initial JDK 11 Build #

📦 independent-projects/resteasy-reactive/server/runtime

geoand commented Oct 20, 2021

Sgitario commented Oct 20, 2021

Sgitario commented Nov 5, 2021

quarkus-bot bot commented Nov 5, 2021 • edited Loading

Failing Jobs - Building eb1994f

Failures

⚙️ Native Tests - Windows - hibernate-validator #

📦 integration-tests/hibernate-validator

geoand commented Nov 29, 2021

Sgitario commented Nov 29, 2021

geoand commented Nov 29, 2021

gsmet commented Jan 12, 2022

geoand commented Jan 12, 2022

FroMage left a comment

Choose a reason for hiding this comment

FroMage commented Jan 18, 2022

Sgitario commented Jan 19, 2022

FroMage left a comment

Choose a reason for hiding this comment

quarkus-bot bot commented Jan 20, 2022 • edited Loading

Failing Jobs - Building 0c102bf

Failures

⚙️ Gradle Tests - JDK 11 Windows #

📦 integration-tests/gradle

quarkus-bot bot commented Jan 20, 2022 • edited Loading

Failing Jobs - Building f154712

Failures

⚙️ Initial JDK 11 Build #

📦 extensions/resteasy-reactive/quarkus-resteasy-reactive/deployment

quarkus-bot bot commented Jan 20, 2022 • edited Loading

Failing Jobs - Building 9f1823d

Failures

⚙️ Gradle Tests - JDK 11 Windows #

📦 integration-tests/gradle

quarkus-bot bot commented Jan 21, 2022

Failing Jobs - Building d684ebe

Failures

⚙️ Gradle Tests - JDK 11 Windows #

📦 integration-tests/gradle

FroMage left a comment

Choose a reason for hiding this comment

FroMage commented Jan 21, 2022

Sgitario commented Oct 20, 2021 •

edited by geoand

Loading

geoand commented Oct 20, 2021 •

edited

Loading

quarkus-bot bot commented Oct 20, 2021 •

edited

Loading

Failing Jobs - Building `1e32c23`

quarkus-bot bot commented Nov 5, 2021 •

edited

Loading

Failing Jobs - Building `eb1994f`

quarkus-bot bot commented Jan 20, 2022 •

edited

Loading

Failing Jobs - Building `0c102bf`

quarkus-bot bot commented Jan 20, 2022 •

edited

Loading

Failing Jobs - Building `f154712`

quarkus-bot bot commented Jan 20, 2022 •

edited

Loading

Failing Jobs - Building `9f1823d`

Failing Jobs - Building `d684ebe`