Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Improve application of masks #10

Merged
merged 5 commits into from
Nov 10, 2023
Merged

Improve application of masks #10

merged 5 commits into from
Nov 10, 2023

Conversation

maawoo
Copy link
Member

@maawoo maawoo commented Nov 10, 2023 •
edited
Loading

  • Load masks with their native dtype
  • Load masks lazily with the same chunks-parameter as the data product.
  • Apply masks on acquisition-chunks first ({'time': 1}) then rechunk to time-chunks ({'time': -1})
  • Use xarray.where instead of Dataset.where as performance seems to be slightly better (Dataset.where performances regression. pydata/xarray#7516 might be relevant)

@maawoo maawoo merged commit 276d743 into main Nov 10, 2023
@maawoo maawoo mentioned this pull request Nov 10, 2023
Closed
@maawoo maawoo deleted the fix_masks branch November 10, 2023 16:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@maawoo