Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Investigate ways to make steal from async-executor more efficient #69

Open
fogti opened this issue Apr 16, 2024 · 2 comments
Open

Investigate ways to make steal from async-executor more efficient #69

fogti opened this issue Apr 16, 2024 · 2 comments

Comments

@fogti
Copy link
Member

fogti commented Apr 16, 2024

It appears that a large amount of time in async-executor is spent shuffling around tasks between different queues. It would be interesting if there are ways to make such stealing of half of a queue faster, perhaps by making such a function part of this crate and utilize knowledge about internals to make such "mass-moves" faster.

@notgull
Copy link
Member

notgull commented Apr 16, 2024

concurrent-queue is ill-suited for this purpose. My plan was to use a dedicated work-stealing queue like crossbeam-deque.

@james7132
Copy link
Contributor

I just attempted this and crossbeam-deque requires an additional function on their Worker or Stealer that supports checking if they're pointing to the same underlying queue (like Arc::ptr_eq) or we risk having a worker steal from itself or failing to remove stealers when dropping a Runner.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

No branches or pull requests

3 participants