Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[dispatcher] Refactor how callbacks are scheduled in the event loop. #11663

Merged
merged 8 commits into from
Jun 24, 2020

Conversation

antoniovicente
Copy link
Contributor

Commit Message: Introduce a separate interface for to schedule callbacks for execution in the event loop. Also migrate existing users of Timer::enableTimer(0ms) that require immediate execution of the callback to use this new interface instead.
Additional Description: This is a functional no-op change in preparation for changes to the behavior of fd activations and 0ms timers. Some event loop callbacks scheduled via 0ms Timers do need to execute in the same event loop iteration they are scheduled in order to provide optimal behavior.
Risk Level: n/a - functional no-op
Testing: unit
Docs Changes: n/a
Release Notes: n/a

…n in the event loop.

Also migrate existing users of Timer::enableTimer(0ms) that require immediate execution of the callback to use this new interface instead.

Signed-off-by: Antonio Vicente <[email protected]>
Copy link
Member

@mattklein123 mattklein123 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice, LGTM with small questions/comments. Thank you!

/wait

include/envoy/event/dispatcher.h Outdated Show resolved Hide resolved
include/envoy/event/schedulable_cb.h Outdated Show resolved Hide resolved
include/envoy/event/schedulable_cb.h Show resolved Hide resolved
/**
* Schedule the callback so it runs in the current iteration of the event loop.
*/
virtual void scheduleCallback() PURE;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is your intention to eventually also handle the 0ms timers that should run in the next/future event loop also via this interface? Will you eventually take an enum here or have a different function for that? Should we add the enum or name this function with that in mind to avoid having to just change it in a follow up?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Eventually I'ld like to get 0ms timers to execute in the next iteration by switching away from event_active as an activation method for 0ms timers. I expect that real timers won't depend on this class.

I do want to use this new interface for fd activations that should happen the next iteration. The relevant code change is staged at: antoniovicente@2194979

The question is if we would prefer scheduleCallback to take a schedule type as an argument or we rather have separate methods for immediate and delayed activations. Use of 2 methods would be less verbose since you wouldn't need to pass in the fully qualified nested enum name. How about something like scheduleCallbackCurrentIteration / scheduleCallbackNextIteration (or scheduleCurrentIteration / scheduleNextIteration)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

2 methods is fine with me. I don't feel strongly about it. Do you want to do the naming in this PR?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm leaning towards changing the name to scheduleCallbackCurrentIteration in this PR, and introducing scheduleCallbackNextIteration in the next PR along with uses and tests.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SGTM.

/wait

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

test/mocks/event/mocks.h Show resolved Hide resolved
Signed-off-by: Antonio Vicente <[email protected]>
Copy link
Contributor Author

@antoniovicente antoniovicente left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the review!

include/envoy/event/schedulable_cb.h Outdated Show resolved Hide resolved
/**
* Schedule the callback so it runs in the current iteration of the event loop.
*/
virtual void scheduleCallback() PURE;
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Eventually I'ld like to get 0ms timers to execute in the next iteration by switching away from event_active as an activation method for 0ms timers. I expect that real timers won't depend on this class.

I do want to use this new interface for fd activations that should happen the next iteration. The relevant code change is staged at: antoniovicente@2194979

The question is if we would prefer scheduleCallback to take a schedule type as an argument or we rather have separate methods for immediate and delayed activations. Use of 2 methods would be less verbose since you wouldn't need to pass in the fully qualified nested enum name. How about something like scheduleCallbackCurrentIteration / scheduleCallbackNextIteration (or scheduleCurrentIteration / scheduleNextIteration)

include/envoy/event/schedulable_cb.h Show resolved Hide resolved
test/mocks/event/mocks.h Show resolved Hide resolved
include/envoy/event/dispatcher.h Outdated Show resolved Hide resolved
@mattklein123
Copy link
Member

Looks like needs a master merge also.

/wait


/**
* Schedule the callback so it runs in the current iteration of the event loop.
*/
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are there any ordering guarantees that this API provides? If so it would be good to document them.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

Note that the semantics for scheduleCallbackCurrentIteration and scheduleCallbackNextIteration (which I will be introducing in the next PR) are very different. scheduleCallbackNextIteration will provide little to no ordering guarantees since the target implementation based on timers at 0ms provides no ordering guarantees (items go through an unordered priority queue)

@yanavlasov yanavlasov self-assigned this Jun 23, 2020
Signed-off-by: Antonio Vicente <[email protected]>
@mattklein123
Copy link
Member

LGTM. Can you check CI?

/wait

@antoniovicente
Copy link
Contributor Author

LGTM. Can you check CI?

/wait

Looks like a pre-existing flake in //test/integration:protocol_integration_test that alyssawilk seems to be looking at after I mentioned it in the envoy-ci slack channel. The CI run before the most recent change to comments was successful.

@antoniovicente
Copy link
Contributor Author

/azp run envoy-presubmit

@azure-pipelines
Copy link

Commenter does not have sufficient privileges for PR 11663 in repo envoyproxy/envoy

Signed-off-by: Antonio Vicente <[email protected]>
@mattklein123 mattklein123 merged commit ca41842 into envoyproxy:master Jun 24, 2020
songhu pushed a commit to songhu/envoy that referenced this pull request Jun 25, 2020
…nvoyproxy#11663)

Introduce a separate interface for to schedule callbacks for execution in the event loop. Also migrate existing users of Timer::enableTimer(0ms) that require immediate execution of the callback to use this new interface instead.

Signed-off-by: Antonio Vicente <[email protected]>
yashwant121 pushed a commit to yashwant121/envoy that referenced this pull request Jul 24, 2020
…nvoyproxy#11663)

Introduce a separate interface for to schedule callbacks for execution in the event loop. Also migrate existing users of Timer::enableTimer(0ms) that require immediate execution of the callback to use this new interface instead.

Signed-off-by: Antonio Vicente <[email protected]>
Signed-off-by: yashwant121 <[email protected]>
mattklein123 pushed a commit that referenced this pull request Aug 6, 2020
…e event loop (#11823)

Processing 0-delay timers in the same loop they are generated can result in long timer callback chains which could starve other operations in the event loop or even result in infinite processing loops. Cases that required same-iteration scheduling behavior for 0-delay timers were refactored to use SchedulableCallback::scheduleCallbackCurrentIteration in #11663, so behavior changes due to this change should be relatively minor.

Signed-off-by: Antonio Vicente <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants