-
Notifications
You must be signed in to change notification settings - Fork 138
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Avoid extra latency potentially happening between Poll
calls
#668
Conversation
a00694e
to
ca45be5
Compare
Yes, the same thought had crossed my mind this weekend. |
@@ -25,4 +26,6 @@ object DiagnosticEvent { | |||
final case class Assigned(partitions: Set[TopicPartition]) extends Rebalance | |||
final case class Lost(partitions: Set[TopicPartition]) extends Rebalance | |||
} | |||
|
|||
final case class RunloopEvent(command: Command) extends DiagnosticEvent |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not a huge fan of this, it means exposing one of the Runloop's internals. Shall we at least document that it's interface is unstable, so we can change the Commands a bit more freely if we need?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Agreed. TBH, I'm not clear what this Diagnostics
thing is for
I assumed it already worked this way, but now I see it doesn't :) |
Not yet, I'm experimenting a bit with some changed polling behavior though. |
Related to work done here:
I didn't manage to find a way to test that this "extra latency" is not stacked anymore 🤔
@svroonland I wonder if there's not a risk of generating/stacking a lot of
Poll
in the Stream 🤔I wonder if using a
Queue.dropping[Command.Poll](1)
wouldn't be a good idea.At any time, I think that we only need to have 1 or 0 Poll command: if we have one, at the next iteration of the loop, it'll consume it from the queue and will execute the corresponding action. If 0, it'll do nothing.
If the queue was containing 1 Poll command and we're emitting a new one, we're replacing the old one with the new one, so we don't waste memory and we have no risks of stacking too many of them (which could potentially lead to an OOM)
WDYT? 🤔