GH-40745: [Java][FlightRPC] Support configuring backpressure threshold #41051
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* Update to grpc-java 1.63.0 * Add to FlightServer.Builder an option to set the number of bytes queued before blocking due to backpressure. Set the default to 10MB instead of gRPC's default of 64K. * Add a ServerInterceptor for automating setting the backpressure threshold on ServerCalls.
jduo
force-pushed
the
40745-flight-java-backpressure-threshold
branch
from
cabd60a to
dd55c79
Compare
|
@github-actions crossbow submit java |
|
Revision: dd55c79 Submitted crossbow builds: ursacomputing/crossbow @ actions-ae0551804f
|
lidavidm
approved these changes
Apr 7, 2024
github-actions
bot
added
awaiting merge
vibhatha
approved these changes
Apr 7, 2024
|
After merging your PR, Conbench analyzed the 4 benchmarking runs that have been run so far on merge-commit 6c14172. There were no benchmark performance regressions. 🎉 The full Conbench report has more details. |
verma-kartik
pushed a commit
to verma-kartik/arrow
that referenced
this pull request
Apr 11, 2024
…reshold (apache#41051) ### Rationale for this change gRPC uses a default backpressure threshold that is too low for services that send large amounts of data such as Arrow Flight. This causes excessive blocking and reduces throughput. ### What changes are included in this PR? * Update to grpc-java 1.63.0 * Add to FlightServer.Builder an option to set the number of bytes queued before blocking due to backpressure. Set the default to 10MB instead of gRPC's default of 64K. * Add a ServerInterceptor for automating setting the backpressure threshold on ServerCalls. ### Are these changes tested? Tested through existing unit tests. ### Are there any user-facing changes? The FlightServer.Builder class has an extra configuration option to let users change the backpressure threshold themselves. * GitHub Issue: apache#40745 Authored-by: James Duong <[email protected]> Signed-off-by: David Li <[email protected]>
tolleybot
pushed a commit
to tmct/arrow
that referenced
this pull request
May 2, 2024
…reshold (apache#41051) ### Rationale for this change gRPC uses a default backpressure threshold that is too low for services that send large amounts of data such as Arrow Flight. This causes excessive blocking and reduces throughput. ### What changes are included in this PR? * Update to grpc-java 1.63.0 * Add to FlightServer.Builder an option to set the number of bytes queued before blocking due to backpressure. Set the default to 10MB instead of gRPC's default of 64K. * Add a ServerInterceptor for automating setting the backpressure threshold on ServerCalls. ### Are these changes tested? Tested through existing unit tests. ### Are there any user-facing changes? The FlightServer.Builder class has an extra configuration option to let users change the backpressure threshold themselves. * GitHub Issue: apache#40745 Authored-by: James Duong <[email protected]> Signed-off-by: David Li <[email protected]>
tolleybot
pushed a commit
to tmct/arrow
that referenced
this pull request
May 4, 2024
…reshold (apache#41051) ### Rationale for this change gRPC uses a default backpressure threshold that is too low for services that send large amounts of data such as Arrow Flight. This causes excessive blocking and reduces throughput. ### What changes are included in this PR? * Update to grpc-java 1.63.0 * Add to FlightServer.Builder an option to set the number of bytes queued before blocking due to backpressure. Set the default to 10MB instead of gRPC's default of 64K. * Add a ServerInterceptor for automating setting the backpressure threshold on ServerCalls. ### Are these changes tested? Tested through existing unit tests. ### Are there any user-facing changes? The FlightServer.Builder class has an extra configuration option to let users change the backpressure threshold themselves. * GitHub Issue: apache#40745 Authored-by: James Duong <[email protected]> Signed-off-by: David Li <[email protected]>
rok
pushed a commit
to tmct/arrow
that referenced
this pull request
May 8, 2024
…reshold (apache#41051) ### Rationale for this change gRPC uses a default backpressure threshold that is too low for services that send large amounts of data such as Arrow Flight. This causes excessive blocking and reduces throughput. ### What changes are included in this PR? * Update to grpc-java 1.63.0 * Add to FlightServer.Builder an option to set the number of bytes queued before blocking due to backpressure. Set the default to 10MB instead of gRPC's default of 64K. * Add a ServerInterceptor for automating setting the backpressure threshold on ServerCalls. ### Are these changes tested? Tested through existing unit tests. ### Are there any user-facing changes? The FlightServer.Builder class has an extra configuration option to let users change the backpressure threshold themselves. * GitHub Issue: apache#40745 Authored-by: James Duong <[email protected]> Signed-off-by: David Li <[email protected]>
rok
pushed a commit
to tmct/arrow
that referenced
this pull request
May 8, 2024
…reshold (apache#41051) ### Rationale for this change gRPC uses a default backpressure threshold that is too low for services that send large amounts of data such as Arrow Flight. This causes excessive blocking and reduces throughput. ### What changes are included in this PR? * Update to grpc-java 1.63.0 * Add to FlightServer.Builder an option to set the number of bytes queued before blocking due to backpressure. Set the default to 10MB instead of gRPC's default of 64K. * Add a ServerInterceptor for automating setting the backpressure threshold on ServerCalls. ### Are these changes tested? Tested through existing unit tests. ### Are there any user-facing changes? The FlightServer.Builder class has an extra configuration option to let users change the backpressure threshold themselves. * GitHub Issue: apache#40745 Authored-by: James Duong <[email protected]> Signed-off-by: David Li <[email protected]>
vibhatha
pushed a commit
to vibhatha/arrow
that referenced
this pull request
May 25, 2024
…reshold (apache#41051) ### Rationale for this change gRPC uses a default backpressure threshold that is too low for services that send large amounts of data such as Arrow Flight. This causes excessive blocking and reduces throughput. ### What changes are included in this PR? * Update to grpc-java 1.63.0 * Add to FlightServer.Builder an option to set the number of bytes queued before blocking due to backpressure. Set the default to 10MB instead of gRPC's default of 64K. * Add a ServerInterceptor for automating setting the backpressure threshold on ServerCalls. ### Are these changes tested? Tested through existing unit tests. ### Are there any user-facing changes? The FlightServer.Builder class has an extra configuration option to let users change the backpressure threshold themselves. * GitHub Issue: apache#40745 Authored-by: James Duong <[email protected]> Signed-off-by: David Li <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Rationale for this change
gRPC uses a default backpressure threshold that is too low for services that send large amounts of data such as Arrow Flight. This causes excessive blocking and reduces throughput.
What changes are included in this PR?
Are these changes tested?
Tested through existing unit tests.
Are there any user-facing changes?
The FlightServer.Builder class has an extra configuration option to let users change the backpressure threshold themselves.