We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
write_csv/json/parquet
Describe the bug
DataFrame::write_csv/json/parquet isn't cancelling safe due to spawned tasks. See these lines: https://github.com/apache/arrow-datafusion/blob/master/datafusion/core/src/physical_plan/file_format/csv.rs#L300-L307
DataFrame::write_csv/json/parquet
It's actually bad because impossible to wrap such functions to timeout or select! for example.
timeout
select!
To Reproduce Wrap it to select or timeout and check that spawned tasks are still working after cancelling by another branch.
select
Expected behavior
It should be safe-to-cancel these methods.
Additional context
Possible ways to fix:
JoinHandle
Drop
abort
Not sure if there are any other spawns which should be covered as well.
spawns
The text was updated successfully, but these errors were encountered:
I can prepare PR to cover this issue if nobody don't mind / already on it 🙂
Sorry, something went wrong.
Successfully merging a pull request may close this issue.
Describe the bug
DataFrame::write_csv/json/parquet
isn't cancelling safe due to spawned tasks. See these lines: https://github.com/apache/arrow-datafusion/blob/master/datafusion/core/src/physical_plan/file_format/csv.rs#L300-L307It's actually bad because impossible to wrap such functions to
timeout
orselect!
for example.To Reproduce
Wrap it to
select
ortimeout
and check that spawned tasks are still working after cancelling by another branch.Expected behavior
It should be safe-to-cancel these methods.
Additional context
Possible ways to fix:
JoinHandle
) to future withDrop
and callabort
for them.Not sure if there are any other
spawns
which should be covered as well.The text was updated successfully, but these errors were encountered: