Buffering of output in Spring Web Reactive with Netty too aggressive [SPR-14943]

[spring-projects-issues]

Daniel Fernández opened SPR-14943 and commented

Scenario

This is the scenario:

• Web application outputting a very large sequence of items produced by a Flux<Item>, serialized as JSON
• Clients expecting to receive first results quickly, then keep on receiving more results until the Flux finishes producing items.
• SSE not being used.

Observed Results

When using Spring Web Reactive in a Spring Boot 2.0.0 (snapshot) application:

• If Netty is used as a server, no results come until the end (Netty seems to buffer all output until it's finished, then send it altogether).
• If RxNetty is used as a server, no results ever come (or at least during a very large period of time).

Compare Tomcat (intro being hit during curl execution to see the data transfer flow):

$ curl http://localhost:8084/items/10000 > out.tomcat
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 1959k    0 1959k    0     0   334k      0 --:--:--  0:00:05 --:--:--  340k
100 2421k    0 2421k    0     0   336k      0 --:--:--  0:00:07 --:--:--  340k

With Netty:

$ curl http://localhost:8082/items/10000 > out.netty
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:--  0:00:05 --:--:--     0
  0     0    0     0    0     0      0      0 --:--:--  0:00:10 --:--:--     0
  0     0    0     0    0     0      0      0 --:--:--  0:00:15 --:--:--     0
  0     0    0     0    0     0      0      0 --:--:--  0:00:20 --:--:--     0
  0     0    0     0    0     0      0      0 --:--:--  0:00:25 --:--:--     0
  0     0    0     0    0     0      0      0 --:--:--  0:00:30 --:--:--     0
100 2421k    0 2421k    0     0  74041      0 --:--:--  0:00:33 --:--:--  579k

And RxNetty:

$ curl http://localhost:8083/items/10000 > out.rxnetty
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:--  0:00:15 --:--:--     0
  0     0    0     0    0     0      0      0 --:--:--  0:00:30 --:--:--     0
  0     0    0     0    0     0      0      0 --:--:--  0:00:45 --:--:--     0
  0     0    0     0    0     0      0      0 --:--:--  0:01:00 --:--:--     0
  0     0    0     0    0     0      0      0 --:--:--  0:01:15 --:--:--     0
  0     0    0     0    0     0      0      0 --:--:--  0:01:30 --:--:--     0
  0     0    0     0    0     0      0      0 --:--:--  0:01:45 --:--:--     0
  0     0    0     0    0     0      0      0 --:--:--  0:02:00 --:--:--     0
  0     0    0     0    0     0      0      0 --:--:--  0:02:15 --:--:--     0
  0     0    0     0    0     0      0      0 --:--:--  0:02:30 --:--:--     0
  0     0    0     0    0     0      0      0 --:--:--  0:02:45 --:--:--     0
... (cancelled after 7 minutes)

Example applications

Example applications: https://github.com/danielfernandez/test-spring-boot-reactive-netty-output

The above applications replicate the scenario using Spring Boot 2.0.0 apps with Jetty, Netty, RxNetty, Tomcat and Undertow. Note this application also tests other issues (to be specified in separate tickets).

Please have a look at the detailed test explanation at the linked repository's README

---

Affects: 5.0 M3

Reference URL: https://github.com/danielfernandez/test-spring-boot-reactive-netty-output

Sub-tasks:

• Add an automatic flushing mechanism for RxNetty [SPR-14991] #19557 Add an automatic flushing mechanism for RxNetty
• Add an integration test to validate Reactor Netty automatic flushing [SPR-14992] #19558 Add an integration test to validate Reactor Netty automatic flushing

Issue Links:

• Add HTTP 1.0 support on Reactor Netty [SPR-14964] #19531 Add HTTP 1.0 support on Reactor Netty ("is duplicated by")
• Large performance impact when using SSE in Spring Web Reactive [SPR-14944] #19511 Large performance impact when using SSE in Spring Web Reactive
• Suboptimal performance of Jetty in Spring Web Reactive compared to Tomcat or Undertow [SPR-14945] #19512 Suboptimal performance of Jetty in Spring Web Reactive compared to Tomcat or Undertow

[spring-projects-issues]

Rossen Stoyanchev commented

Note that Flux<Item> currently produces a JSON array which is what browser clients can consume. In that sense it's not designed for a streaming scenario as opposed to SSE which enforces a flush after each event. That said we should consider improvements:

Why does it work better in Tomcat/Undertow out of the box and can we ensure similar reasonable defaults across all runtimes?

How do we support outputting a very large sequence of items other than SSE? Ideally returning Flux<Item> should support streaming and non-streaming scenarios just as well so let's start from there and see if that is possible.

[spring-projects-issues]

Sébastien Deleuze commented

For Reactor Netty, it should maybe flush the buffer when a specific amount of data has been sent, I am going to discuss that with Reactor team.
With RxNetty, 184 iterations works while 185 does not! I am going to try to reproduce it with a pure RxNetty application.

[spring-projects-issues]

Sébastien Deleuze commented

Reactor Netty will introduce a threshold to trigger a flush after buffering a certain amount of data, see this issue.

[spring-projects-issues]

Sébastien Deleuze commented

I have been able to reproduce this issue with a pure RxNetty sample application and have raised an issue on RxNetty bugtracker.

[spring-projects-issues]

Sébastien Deleuze commented

The 2 subtasks have been resolved, I am just waiting a fix from Stéphane Maldini that will allow to restore Reactor Netty based integration tests (currently disabled via this commit).