Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add non blocking Apache HttpClient 5 based Transport. #1136

Merged

Conversation

maciejwalkowiak
Copy link
Contributor

📜 Description

Add non blocking Apache HttpClient 5 based Transport suitable for high traffic backend applications.

💡 Motivation and Context

Current AsyncTransport can perform up to 3 requests per second which is too little for performance feature running in backend applications.

💚 How did you test it?

Unit tests plus sample app for running performance tests.

📝 Checklist

  • I reviewed submitted code
  • I added tests to verify changes
  • All tests passing
  • No breaking changes

🔮 Next steps

Auto-configure new transport in Spring Boot auto-configuration when apache http client is on the classpath.

@maciejwalkowiak
Copy link
Contributor Author

@marandaneto can you take a look what's the problem with Android sample here? There cannot be optional dependencies for the sentry module?

@marandaneto
Copy link
Contributor

@marandaneto can you take a look what's the problem with Android sample here? There cannot be optional dependencies for the sentry module?

see my comments, I bet they are not Android compatible.
I'm on vacation till the 4th so can't look into it in detail right now.

@maciejwalkowiak
Copy link
Contributor Author

Thanks @marandaneto. Merry Christmas and see you on the 4th!

@codecov-io
Copy link

codecov-io commented Dec 29, 2020

Codecov Report

Merging #1136 (60f95c1) into gh-1097-rate-limiter (28a0068) will decrease coverage by 0.02%.
The diff coverage is 79.77%.

Impacted file tree graph

@@                    Coverage Diff                     @@
##             gh-1097-rate-limiter    #1136      +/-   ##
==========================================================
- Coverage                   74.91%   74.88%   -0.03%     
- Complexity                   1613     1624      +11     
==========================================================
  Files                         169      170       +1     
  Lines                        5617     5706      +89     
  Branches                      548      557       +9     
==========================================================
+ Hits                         4208     4273      +65     
- Misses                       1153     1172      +19     
- Partials                      256      261       +5     
Impacted Files Coverage Δ Complexity Δ
...ry/transport/apache/ApacheHttpClientTransport.java 72.30% <72.30%> (ø) 8.00 <8.00> (?)
...sport/apache/ApacheHttpClientTransportFactory.java 100.00% <100.00%> (ø) 3.00 <3.00> (?)
...io/sentry/spring/boot/SentryAutoConfiguration.java 93.87% <100.00%> (+0.26%) 1.00 <0.00> (ø)
...main/java/io/sentry/AsyncHttpTransportFactory.java 100.00% <100.00%> (ø) 2.00 <1.00> (ø)
...ava/io/sentry/SentryEnvelopeItemHeaderAdapter.java 64.28% <0.00%> (-7.15%) 7.00% <0.00%> (-1.00%)
...in/java/io/sentry/SentryEnvelopeHeaderAdapter.java 75.58% <0.00%> (-3.49%) 17.00% <0.00%> (-1.00%)
sentry/src/main/java/io/sentry/GsonSerializer.java 96.77% <0.00%> (-0.20%) 9.00% <0.00%> (ø%)
...a/io/sentry/exception/SentryEnvelopeException.java
... and 2 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 28a0068...60f95c1. Read the comment docs.

@maciejwalkowiak
Copy link
Contributor Author

@marandaneto @bruno-garcia please find some time to review this PR.

@bruno-garcia
Copy link
Member

@maciejwalkowiak Manoel is back next week.

It LGTM but I have a note on the stream->byte array and the json type of the body, lets discuss that before merging

@@ -13,6 +13,8 @@
* Ref: `ITransport` implementations are now responsible for executing request in asynchronous or synchronous way (#1118)
* Ref: Add option to set `TransportFactory` instead of `ITransport` on `SentryOptions` (#1124)
* Ref: Simplify ITransport creation in ITransportFactory (#1135)
* Feat: Add non blocking Apache HttpClient 5 based Transport (#1136)
* Enhancement: Autoconfigure Apache HttpClient 5 based Transport in Spring Boot integration (#1143)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is part of the PR template next steps, but it's in the changelog as well, which one would be not up to date?

@@ -57,6 +57,8 @@ object Config {
val springAop = "org.springframework:spring-aop"
val aspectj = "org.aspectj:aspectjweaver"
val servletApi = "javax.servlet:javax.servlet-api"

val apacheHttpClient = "org.apache.httpcomponents.client5:httpclient5:5.0.3"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1st stable version of this lib v5 appeared in Feb, 2020, do you think that the majority of Apps have been upgraded already?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is no package collision between version 5 and 4 so even if they did not, it means just an extra dependency. We still can add transport based on 4.x if there will be such request.

new FutureCallback<SimpleHttpResponse>() {
@Override
public void completed(SimpleHttpResponse response) {
currentlyRunning.decrementAndGet();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should this be the last line to be called? theoretically there's still something happening.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is but it means that the connection is not occupied anymore. I don't see any risk here.

Comment on lines 78 to 80
request.setHeader("Content-Encoding", "gzip");
request.setHeader("Content-Type", "application/x-sentry-envelope");
request.setHeader("Accept", "application/json");
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this be part of RequestDetailsResolver or even extracting to a new class? it's a duplicated code right now.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If it's just 3 lines of code I wouldn't bother

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if RequestDetailsResolver is used in both transports and they are meant to configure the connection with its headers, I don't see why duplicating it though.
https://github.com/getsentry/sentry-java/blob/gh-1097-apache-non-blocking-transport/sentry/src/main/java/io/sentry/transport/HttpConnection.java#L124-L126

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's up to the transport to decide if it sends gzip or not, I also don't think that we should set fixed accept header. RequestDetailsResolver returns headers that users MUST send to Sentry.

@maciejwalkowiak
Copy link
Contributor Author

@bruno-garcia @marandaneto take another look please.

@maciejwalkowiak maciejwalkowiak merged commit 92876c8 into gh-1097-rate-limiter Jan 12, 2021
@maciejwalkowiak maciejwalkowiak deleted the gh-1097-apache-non-blocking-transport branch January 12, 2021 14:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants