-
Notifications
You must be signed in to change notification settings - Fork 1.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Improve documentation (and ASCII art) about streaming execution, and thread pools #13423
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM, the ASCII art is really nice. I left some suggestions for minor typos.
Co-authored-by: Jonah Gao <[email protected]>
…nto alamb/doc_threadpools
Co-authored-by: Jonah Gao <[email protected]>
…nto alamb/doc_threadpools
The explanation for the problem is super clear One thing I don't understand is: For Tokio's convention, all tasks created by
|
I took a quick look at this and content looks good, couldn't check diagrams as on phone.
We could, however, to preserve the thread per core architecture we would need to cap the threads of the blocking pool to the core count. Then as blocking tasks can't yield waiting for input, every CPU bound morsel would have to be spawned separately. This would give us a "morsel-driven" scheduler, however, tokio has a relatively high per task overhead and so even discounting the sheer amount of boilerplate this would require, the performance would be regrettable. If going down this path you might as well switch to an actual morsel driven scheduler (although this is at this stage likely intractable). Ultimately spawn_blocking is designed for blocking IO, it is not designed for CPU bound tasks. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM👍, thanks @alamb
Thank you @2010YOUY01 for the question and @tustvold for the answer -- this question also has come up in the past so I added a summary in ad16179 |
//! applications as explained in the [Using Rustlang’s Async Tokio | ||
//! Runtime for CPU-Bound Tasks] blog. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should this be linking to https://thenewstack.io/using-rustlangs-async-tokio-runtime-for-cpu-bound-tasks/
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
yes -- I think it does (line 584)
//! During execution, DataFusion creates this many distinct `async` [`Stream`]s and | ||
//! this many distinct [Tokio] [`task`]s, which drive the `Stream`s |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I was aware that target_partitions
and number of Stream
s matched the number of CPU cores but I was not aware that we cap the number of Tokio tasks as well. Can you expand on that point / are we getting all the benefits of the tokio runtime (work stealing, etc) by keeping that number (relatively) low? for context the local task queue size per thread is 256 tasks
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think it is somewhat nuanced -- what happens is that certain operations (like RepartionExec
spawn tasks to read from the inputs. As written I now see this sentence is somewhat misleading. I will try and clarify in a follow on PR
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Tried to clarify in #13474
Thank you everyone for the comments -- I am going to merge this one in and make some adjustments as a follow on PR to avoid keeping it open for too long. Long live comments! |
Which issue does this PR close?
Rationale for this change
In order to understand the need for a second runtime, I needed to write up the background material so it made sense.
Also, this is starting to come up with dft:
DedicatedExecutor
to FlightSQL Server datafusion-contrib/datafusion-dft#247What changes are included in this PR?
This PR adds background documentation on how DataFusion runs plans and why a separate Runtime may be needed to keep the network busy
Note I am also working on an example to show how to actually use a second runtime which I will link to these docs when it is ready
Also, I found myself on a✈️ without WIFI so I also made a bunch of ASCII art while I was at it)
Are these changes tested?
By CI
Are there any user-facing changes?
Nice documentation hopefully!