Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support parallel reads using dask.delayed #48

Open
wants to merge 22 commits into
base: main
Choose a base branch
from
Open

Support parallel reads using dask.delayed #48

wants to merge 22 commits into from

Conversation

huard
Copy link
Collaborator

@huard huard commented May 24, 2023

  • Include documentation when adding new features.
  • Include new tests or update existing tests when applicable.

When parallel is True, open_dataset is called using dask.delayed.

@huard huard requested a review from aulemahal May 24, 2023 20:02
Copy link
Contributor

@aulemahal aulemahal left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry for the delay, this had fallen in some cracks.

Looks good. Do you want to precise the level of parallelization in the docstring of open_ncml ? (Like that aggregations are individually processed in parallel ? ) Or is it rare that a ncml has more than one "Aggregation" node ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants