-
Notifications
You must be signed in to change notification settings - Fork 14k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
MINOR: Remove throttling logic from RecordAccumulator #7195
MINOR: Remove throttling logic from RecordAccumulator #7195
Conversation
One thing I don't understand is why this logic only runs if max.in.flight=1. cc @lbradstreet |
eca22b2
to
3a137c5
Compare
@@ -588,13 +587,13 @@ private void handleProduceResponse(ClientResponse response, Map<TopicPartition, | |||
TopicPartition tp = entry.getKey(); | |||
ProduceResponse.PartitionResponse partResp = entry.getValue(); | |||
ProducerBatch batch = batches.get(tp); | |||
completeBatch(batch, partResp, correlationId, now, receivedTimeMs + produceResponse.throttleTimeMs()); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is the idea that we can remove the throttling here, and rely on the existing NetworkClient throttle instead?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes. This logic was only active for max.inflight=1 which is confusing and seemingly unnecessary. We have logic to remove nodes from readyNodes
if the client is not ready.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sounds good. It does look pretty unnecessary to me too.
Will fix the checkstyle error. |
@junrao Does this look good to you? |
Retest this please. One build passed, two timed out. |
The current code works, but it's a bit wasteful and potentially confusing.
ad079de
to
f9ae778
Compare
One job passed and the other failed with two known flakes:
@rajinisivaram does this look good to you? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@ijuma Thanks for the PR, LGTM
This is redundant since
Sender
andNetworkClient
handle throttling. It'salso confusing since the
RecordAccumulator
logic only applies whenmax.in.flight.requests.per.connection=1
.In
Sender.sendProducerData
, the following code handles throttling:Committer Checklist (excluded from commit message)