Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Better handling for blocking tasks #545

Open
morgante opened this issue Oct 21, 2024 · 1 comment
Open

Better handling for blocking tasks #545

morgante opened this issue Oct 21, 2024 · 1 comment

Comments

@morgante
Copy link
Contributor

When we pick up a slow file (primarily because it invokes JavaScript callback or other large overhead) we want this to only block that thread.

It's not a huge problem to block one thread at a time since the ratio if slow work to fast work is usually fine. The problem is when we allow fast work to queue up on that same thread.

@morgante
Copy link
Contributor Author

In #546 I investigated and if some tasks are much slower, it looks like spawn is a meaningful improvement over par_iter.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant