fix(3.2): Triple Reactor OneToMany Handler null pointer fix and DubboFilter support #14125

caoyanan666 · 2024-04-23T10:05:15Z

What is the purpose of the change

AbstractTripleReactorSubscriber initializes downstream only upon calling the subscribe method. Consequently, if an onError event happens before subscription, it can result in a null pointer, causing the request to hang. For instance, if there's an exception during parameter validation in a request, this would trigger onError directly without Flux reaching the subscribe step.
The invoke method of OneToManyMethodHandler directly returns a CompletableFuture.completedFuture(null), bypassing DubboFilter. For instance, if I have a DubboAPMFilter, it monitors the response time of CompletableFuture.completedFuture(null) as 0 milliseconds. Hence, in ServerTripleReactorSubscriber, I maintain a CompletableFuture to return the actual execution Future to the Invoker instead of returning a fake CompletableFuture.completedFuture(null).

Brief changelog

Verifying this change

Checklist

Make sure there is a GitHub_issue field for the change (usually before you start working on it). Trivial changes like typos do not require a GitHub issue. Your pull request should address just this issue, without pulling in other changes - one PR resolves one issue.
Each commit in the pull request should have a meaningful subject line and body.
Write a pull request description that is detailed enough to understand what the pull request does, how, and why.
Check if is necessary to patch to Dubbo 3 if you are work on Dubbo 2.7
Write necessary unit-test to verify your logic correction, more mock a little better when cross module dependency exist. If the new feature or significant change is committed, please remember to add sample in dubbo samples project.
Add some description to dubbo-website project if you are requesting to add a feature.
GitHub Actions works fine on your own branch.
If this contribution is large, please follow the Software Donation Guide.

…port

caoyanan666 · 2024-04-24T05:37:37Z

@AlbumenJ @icodening @EarthChen PTAL

AlbumenJ · 2024-05-08T06:30:23Z

@caoyanan666 Please fix the conflicts

# Conflicts: # dubbo-plugin/dubbo-reactive/src/main/java/org/apache/dubbo/reactive/calls/ReactorServerCalls.java

caoyanan666 · 2024-05-09T03:50:51Z

@AlbumenJ Does the unit test code have a GitHub project link? I want to run it locally to give it a try.

caoyanan666 · 2024-05-09T04:14:51Z

https://github.com/apache/dubbo/actions/runs/9011837780/job/24760283649?pr=14125
这个13787失败我排查到是因为超时
[12/23] [dubbo-samples-test-13787:1/1] TEST FAILURE: Run tests timeout, version: -Ddubbo.version=3.2.13-SNAPSHOT -Dspring-boot.version=2.7.6 -Djava.version=8, please check logs: /home/runner/work/dubbo/dubbo/99-integration/dubbo-samples-test-13787/target/logs
然后继续排查发现因为provider容器异常退出
dependency failed to start: container dubbo-samples-test-13787--dubbo-samples-test-13787-provider-1 exited (0)
Redirect container logs: dubbo-samples-test-13787--dubbo-samples-test-13787-provider-1
然后看容器inspect显示没有监听到20800端口
"Output": "checking tcp ports: 127.0.0.1:20880;, start at: 0, timeout: 1\nchecking tcp port [127.0.0.1:20880] ...\ntelnet: Unable to connect to remote host: Connection refused\nTrying 127.0.0.1...\ncheck tcp port [127.0.0.1:20880] is timeout: 1 s\ncheck ports failure\n"
然后我看服务启动日志是正常启动后直接就shutdown了，这个是为啥
2024-05-09 03:33:08.630 INFO 16 --- [ main] org.apache.dubbo.metadata.MetadataInfo : [DUBBO] metadata revision changed: null -> cc997f680e96066e3e0d4ecd5d38730f, app: provider, services: 1, dubbo version: 3.2.13-SNAPSHOT, current host: 172.27.0.3
2024-05-09 03:33:08.744 INFO 16 --- [ main] o.a.dubbo.samples.test.DubboProvider : Started DubboProvider in 4.473 seconds (JVM running for 5.559)
dubbo service started
2024-05-09 03:33:08.752 INFO 16 --- [ionShutdownHook] o.a.d.c.deploy.DefaultModuleDeployer : [DUBBO] Dubbo Module[1.1.1] is stopping., dubbo version: 3.2.13-SNAPSHOT, current host: 172.27.0.3
@AlbumenJ 帮忙给个排查思路

caoyanan666 · 2024-05-09T05:48:59Z

https://github.com/apache/dubbo/actions/runs/9011837780/job/24760283649?pr=14125 这个13837失败我排查到是因为超时 [12/23] [dubbo-samples-test-13787:1/1] TEST FAILURE: Run tests timeout, version: -Ddubbo.version=3.2.13-SNAPSHOT -Dspring-boot.version=2.7.6 -Djava.version=8, please check logs: /home/runner/work/dubbo/dubbo/99-integration/dubbo-samples-test-13787/target/logs 然后继续排查发现因为provider容器异常退出 dependency failed to start: container dubbo-samples-test-13787--dubbo-samples-test-13787-provider-1 exited (0) Redirect container logs: dubbo-samples-test-13787--dubbo-samples-test-13787-provider-1 然后看容器inspect显示没有监听到20800端口 "Output": "checking tcp ports: 127.0.0.1:20880;, start at: 0, timeout: 1\nchecking tcp port [127.0.0.1:20880] ...\ntelnet: Unable to connect to remote host: Connection refused\nTrying 127.0.0.1...\ncheck tcp port [127.0.0.1:20880] is timeout: 1 s\ncheck ports failure\n" 然后我看服务启动日志是正常启动后直接就shutdown了，这个是为啥 2024-05-09 03:33:08.630 INFO 16 --- [ main] org.apache.dubbo.metadata.MetadataInfo : [DUBBO] metadata revision changed: null -> cc997f680e96066e3e0d4ecd5d38730f, app: provider, services: 1, dubbo version: 3.2.13-SNAPSHOT, current host: 172.27.0.3 2024-05-09 03:33:08.744 INFO 16 --- [ main] o.a.dubbo.samples.test.DubboProvider : Started DubboProvider in 4.473 seconds (JVM running for 5.559) dubbo service started 2024-05-09 03:33:08.752 INFO 16 --- [ionShutdownHook] o.a.d.c.deploy.DefaultModuleDeployer : [DUBBO] Dubbo Module[1.1.1] is stopping., dubbo version: 3.2.13-SNAPSHOT, current host: 172.27.0.3 @AlbumenJ 帮忙给个排查思路

这个原因有可能是端口起在了20881上，具体为啥20880没启动成功，日志里面定位不到
2024-05-09 03:33:08.478 WARN 16 --- [ main] org.apache.dubbo.config.ServiceConfig : [DUBBO] Use random available port(20881) for protocol dubbo, dubbo version: 3.2.13-SNAPSHOT, current host: 172.27.0.3, error code: 5-8. This may be caused by , go to https://dubbo.apache.org/faq/5/8 to find instructions.

caoyanan666 · 2024-05-09T06:31:13Z

https://github.com/apache/dubbo/actions/runs/9011837780/job/24760283649?pr=14125 这个13837失败我排查到是因为超时 [12/23] [dubbo-samples-test-13787:1/1] TEST FAILURE: Run tests timeout, version: -Ddubbo.version=3.2.13-SNAPSHOT -Dspring-boot.version=2.7.6 -Djava.version=8, please check logs: /home/runner/work/dubbo/dubbo/99-integration/dubbo-samples-test-13787/target/logs 然后继续排查发现因为provider容器异常退出 dependency failed to start: container dubbo-samples-test-13787--dubbo-samples-test-13787-provider-1 exited (0) Redirect container logs: dubbo-samples-test-13787--dubbo-samples-test-13787-provider-1 然后看容器inspect显示没有监听到20800端口 "Output": "checking tcp ports: 127.0.0.1:20880;, start at: 0, timeout: 1\nchecking tcp port [127.0.0.1:20880] ...\ntelnet: Unable to connect to remote host: Connection refused\nTrying 127.0.0.1...\ncheck tcp port [127.0.0.1:20880] is timeout: 1 s\ncheck ports failure\n" 然后我看服务启动日志是正常启动后直接就shutdown了，这个是为啥 2024-05-09 03:33:08.630 INFO 16 --- [ main] org.apache.dubbo.metadata.MetadataInfo : [DUBBO] metadata revision changed: null -> cc997f680e96066e3e0d4ecd5d38730f, app: provider, services: 1, dubbo version: 3.2.13-SNAPSHOT, current host: 172.27.0.3 2024-05-09 03:33:08.744 INFO 16 --- [ main] o.a.dubbo.samples.test.DubboProvider : Started DubboProvider in 4.473 seconds (JVM running for 5.559) dubbo service started 2024-05-09 03:33:08.752 INFO 16 --- [ionShutdownHook] o.a.d.c.deploy.DefaultModuleDeployer : [DUBBO] Dubbo Module[1.1.1] is stopping., dubbo version: 3.2.13-SNAPSHOT, current host: 172.27.0.3 @AlbumenJ 帮忙给个排查思路

这个原因有可能是端口起在了20881上，具体为啥20880没启动成功，日志里面定位不到 2024-05-09 03:33:08.478 WARN 16 --- [ main] org.apache.dubbo.config.ServiceConfig : [DUBBO] Use random available port(20881) for protocol dubbo, dubbo version: 3.2.13-SNAPSHOT, current host: 172.27.0.3, error code: 5-8. This may be caused by , go to https://dubbo.apache.org/faq/5/8 to find instructions.

好像是3.2所有的ci都卡在这了， https://github.com/apache/dubbo/actions/runs/9011299521/job/24758927418?pr=14166 ，坐等修复...

AlbumenJ · 2024-05-10T02:18:54Z

Try to fix it in apache/dubbo-integration-cases#25

AlbumenJ · 2024-05-10T02:18:59Z

You can ignore it

caoyanan666 · 2024-05-10T07:29:57Z

@AlbumenJ PTAL

CrazyHZM · 2024-05-10T17:01:31Z

.../dubbo-reactive/src/main/java/org/apache/dubbo/reactive/AbstractTripleReactorSubscriber.java

@@ -53,7 +53,7 @@ public void subscribe(final CallStreamObserver<T> downstream) {
        if (downstream == null) {
            throw new NullPointerException();
        }
-        if (this.downstream == null && SUBSCRIBED.compareAndSet(false, true)) {
+        if (SUBSCRIBED.compareAndSet(false, true)) {


Please help optimize the code style. Thanks.

I ran spotless:apply again. Do you mean the hard coding of true and false?

CrazyHZM · 2024-05-10T17:05:34Z

...in/dubbo-reactive/src/main/java/org/apache/dubbo/reactive/ServerTripleReactorSubscriber.java

 /**
 * The Subscriber in server to passing the data produced by user publisher to responseStream.
 */
 public class ServerTripleReactorSubscriber<T> extends AbstractTripleReactorSubscriber<T> {

+    private final List<T> collectedData = new ArrayList<>();
+    private final CompletableFuture<List<T>> completableFuture = new CompletableFuture<>();


Make naming completableFuture more meaningful.

Why is List type? Is combined future?

sonarcloud · 2024-05-11T03:13:15Z

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
82.1% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarCloud

EarthChen

LGTM

caoyanan added 4 commits April 7, 2024 17:18

ReactorDubbo3TripleStub.mustache add schema registry

3632b1e

Triple Reactor OneToMany Handler null pointer fix and DubboFilter sup…

a124287

…port

trigger ci

fe0cb95

Adjust the order

cb80332

Merge branch '3.2' into 3.2

63eef76

caoyanan666 and others added 3 commits May 9, 2024 10:33

Merge branch 'apache:3.2' into 3.2

f79b6df

Merge branch '3.2' into fix-3.2-triple-reactor-one-to-many

8c59784

# Conflicts: # dubbo-plugin/dubbo-reactive/src/main/java/org/apache/dubbo/reactive/calls/ReactorServerCalls.java

trigger ci

bd296ee

caoyanan666 closed this May 9, 2024

caoyanan666 reopened this May 9, 2024

CrazyHZM requested changes May 10, 2024

View reviewed changes

ServerTripleReactorSubscriber meaningful

924f840

caoyanan666 requested a review from CrazyHZM May 11, 2024 02:55

EarthChen approved these changes May 11, 2024

View reviewed changes

EarthChen merged commit 5d1c0ea into apache:3.2 May 11, 2024
19 checks passed

CrazyHZM added this to the 3.2.13 milestone May 12, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix(3.2): Triple Reactor OneToMany Handler null pointer fix and DubboFilter support #14125

fix(3.2): Triple Reactor OneToMany Handler null pointer fix and DubboFilter support #14125

caoyanan666 commented Apr 23, 2024 •

edited

Loading

caoyanan666 commented Apr 24, 2024

AlbumenJ commented May 8, 2024

caoyanan666 commented May 9, 2024

caoyanan666 commented May 9, 2024 •

edited

Loading

caoyanan666 commented May 9, 2024

caoyanan666 commented May 9, 2024

AlbumenJ commented May 10, 2024

AlbumenJ commented May 10, 2024

caoyanan666 commented May 10, 2024

CrazyHZM May 10, 2024

caoyanan666 May 11, 2024

CrazyHZM May 10, 2024

sonarcloud bot commented May 11, 2024

EarthChen left a comment

fix(3.2): Triple Reactor OneToMany Handler null pointer fix and DubboFilter support #14125

fix(3.2): Triple Reactor OneToMany Handler null pointer fix and DubboFilter support #14125

Conversation

caoyanan666 commented Apr 23, 2024 • edited Loading

What is the purpose of the change

Brief changelog

Verifying this change

Checklist

caoyanan666 commented Apr 24, 2024

AlbumenJ commented May 8, 2024

caoyanan666 commented May 9, 2024

caoyanan666 commented May 9, 2024 • edited Loading

caoyanan666 commented May 9, 2024

caoyanan666 commented May 9, 2024

AlbumenJ commented May 10, 2024

AlbumenJ commented May 10, 2024

caoyanan666 commented May 10, 2024

CrazyHZM May 10, 2024

Choose a reason for hiding this comment

caoyanan666 May 11, 2024

Choose a reason for hiding this comment

CrazyHZM May 10, 2024

Choose a reason for hiding this comment

sonarcloud bot commented May 11, 2024

Quality Gate passed

EarthChen left a comment

Choose a reason for hiding this comment

caoyanan666 commented Apr 23, 2024 •

edited

Loading

caoyanan666 commented May 9, 2024 •

edited

Loading