Per-variable specification of boolean parameters in open_dataset #9218

Ostheer · 2024-07-08T13:42:17Z

Added feature to choose whether to apply mask_and_scale for each DataArray in a Dataset individually.

The mask_and_scale parameter can now not only be bool, but also a mapping from variable name (str) to bool.

- [?] User visible changes (including notable bug fixes) are documented in whats-new.rst

New functions/methods are listed in api.rst

I wasn't sure if I was supposed to edit whats-new.rst, so I didn't touch it.

for more information, see https://pre-commit.ci

Ostheer · 2024-07-12T12:25:39Z

Can someone tell me if and how I should edit the what's new file?

I could add something like

- Allow per-variable specification of `mask_and_scale` in `open_dataset`  (:pull:`9218`).
  By `Mathijs Verhaegh <https://github.com/Ostheer>`_.

But I'm not sure where, as I don't know into which upcoming release this commit would be included.

max-sixty · 2024-07-12T16:25:53Z

This looks good!

Great re whatsnew suggestion, within New Features works!

xarray/backends/api.py

xarray/conventions.py

headtr1ck · 2024-07-12T22:39:03Z

Could also easily extend to other arguments, like decode_times. What do you think?

Co-authored-by: Michael Niklas <[email protected]>

Otherwise you lose all typing when you use that because it returns Any.

for more information, see https://pre-commit.ci

Ostheer · 2024-07-12T23:09:04Z

Could also easily extend to other arguments, like decode_times. What do you think?

Added for 4 more params (46de50f)

Feature now affects:

mask_and_scale, decode_times, decode_timedelta, use_cftime, concat_characters

headtr1ck

Add an entry in what's new and this is good! Thanks

xarray/conventions.py

for more information, see https://pre-commit.ci

…nto granular-mask-and-scale

for more information, see https://pre-commit.ci

Ostheer · 2024-07-13T11:38:48Z

I resolved some minor Mypy issues stemming from the annotations on _item_or_default, but the Mypy and Mypy 3.9 tasks are still failing.

I can't find how the errors in the logs relate to any of the code changes. Any ideas?

headtr1ck · 2024-07-13T12:24:19Z

I resolved some minor Mypy issues stemming from the annotations on _item_or_default, but the Mypy and Mypy 3.9 tasks are still failing.

I can't find how the errors in the logs relate to any of the code changes. Any ideas?

These checks are failing on main as well. There are already some PRs open to fix these.

I would say this looks good.

Ostheer · 2024-07-13T12:40:38Z

I resolved some minor Mypy issues stemming from the annotations on _item_or_default, but the Mypy and Mypy 3.9 tasks are still failing.
I can't find how the errors in the logs relate to any of the code changes. Any ideas?

These checks are failing on main as well. There are already some PRs open to fix these.

I would say this looks good.

I see. Can we then somehow merge this despite the failing checks? I don't think Github lets me.

headtr1ck · 2024-07-13T13:40:44Z

Yes, no worries.
Let's keep it a day or two for other people to check and then merge. Thanks!

xarray/backends/api.py

max-sixty · 2024-07-16T06:25:36Z

Thanks @Ostheer !

welcome · 2024-07-16T06:25:43Z

Congratulations on completing your first pull request! Welcome to Xarray! We are proud of you, and hope to see you again!

* main: Enable pandas type checking (pydata#9213) Per-variable specification of boolean parameters in open_dataset (pydata#9218) test push Added a space to the documentation (pydata#9247) Fix typing for test_plot.py (pydata#9234) Allow mypy to run in vscode (pydata#9239) Revert "Test main push" Test main push Revert "Update _typing.py" Update _typing.py Add a `.drop_attrs` method (pydata#8258)

* main: add backend intro and how-to diagram (#9175) Fix copybutton for multi line examples in double digit ipython cells (#9264) Update signature for _arrayfunction.__array__ (#9237) Add encode_cf_datetime benchmark (#9262) groupby, resample: Deprecate some positional args (#9236) Delete ``base`` and ``loffset`` parameters to resample (#9233) Update dropna docstring (#9257) Grouper, Resampler as public api (#8840) Fix mypy on main (#9252) fix fallback isdtype method (#9250) Enable pandas type checking (#9213) Per-variable specification of boolean parameters in open_dataset (#9218) test push Added a space to the documentation (#9247) Fix typing for test_plot.py (#9234)

* main: (54 commits) Adding `open_datatree` backend-specific keyword arguments (#9199) [pre-commit.ci] pre-commit autoupdate (#9202) Restore ability to specify _FillValue as Python native integers (#9258) add backend intro and how-to diagram (#9175) Fix copybutton for multi line examples in double digit ipython cells (#9264) Update signature for _arrayfunction.__array__ (#9237) Add encode_cf_datetime benchmark (#9262) groupby, resample: Deprecate some positional args (#9236) Delete ``base`` and ``loffset`` parameters to resample (#9233) Update dropna docstring (#9257) Grouper, Resampler as public api (#8840) Fix mypy on main (#9252) fix fallback isdtype method (#9250) Enable pandas type checking (#9213) Per-variable specification of boolean parameters in open_dataset (#9218) test push Added a space to the documentation (#9247) Fix typing for test_plot.py (#9234) Allow mypy to run in vscode (#9239) Revert "Test main push" ...

…monotonic-variable * main: (995 commits) Adding `open_datatree` backend-specific keyword arguments (pydata#9199) [pre-commit.ci] pre-commit autoupdate (pydata#9202) Restore ability to specify _FillValue as Python native integers (pydata#9258) add backend intro and how-to diagram (pydata#9175) Fix copybutton for multi line examples in double digit ipython cells (pydata#9264) Update signature for _arrayfunction.__array__ (pydata#9237) Add encode_cf_datetime benchmark (pydata#9262) groupby, resample: Deprecate some positional args (pydata#9236) Delete ``base`` and ``loffset`` parameters to resample (pydata#9233) Update dropna docstring (pydata#9257) Grouper, Resampler as public api (pydata#8840) Fix mypy on main (pydata#9252) fix fallback isdtype method (pydata#9250) Enable pandas type checking (pydata#9213) Per-variable specification of boolean parameters in open_dataset (pydata#9218) test push Added a space to the documentation (pydata#9247) Fix typing for test_plot.py (pydata#9234) Allow mypy to run in vscode (pydata#9239) Revert "Test main push" ...

Mathijs Verhaegh and others added 4 commits July 8, 2024 15:24

allow per-variable choice of mask_and_scale in open_dataset

e63d15c

simplify docstring datatype

4bbb924

[pre-commit.ci] auto fixes from pre-commit.com hooks

fa90e7f

for more information, see https://pre-commit.ci

Merge branch 'pydata:main' into granular-mask-and-scale

b70f00f

max-sixty added the plan to merge Final call for comments label Jul 12, 2024

headtr1ck reviewed Jul 12, 2024

View reviewed changes

xarray/backends/api.py Outdated Show resolved Hide resolved

xarray/conventions.py Outdated Show resolved Hide resolved

Ostheer and others added 4 commits July 13, 2024 00:39

dict -> Mapping in type annotation

b218b3c

Co-authored-by: Michael Niklas <[email protected]>

use typevar for _item_or_default annotation

72483cb

Otherwise you lose all typing when you use that because it returns Any.

[pre-commit.ci] auto fixes from pre-commit.com hooks

a47363c

for more information, see https://pre-commit.ci

implement feature for 4 additional parameters

46de50f

Ostheer changed the title ~~Granular mask and scale~~ Per-variable specification of boolean parameters in open_dataset Jul 12, 2024

fix default value inconsistency

28e01ab

headtr1ck approved these changes Jul 13, 2024

View reviewed changes

xarray/conventions.py Outdated Show resolved Hide resolved

Mathijs Verhaegh and others added 7 commits July 13, 2024 12:05

add what's new + None annotation

a194d0d

Merge branch 'main' into granular-mask-and-scale

83dfcc1

[pre-commit.ci] auto fixes from pre-commit.com hooks

0f1e432

for more information, see https://pre-commit.ci

_item_or_default return type T | None

5eecad1

Merge branch 'granular-mask-and-scale' of github.com:Ostheer/xarray i…

11a60e3

…nto granular-mask-and-scale

remove deault default value _item_or_default

bc664f6

[pre-commit.ci] auto fixes from pre-commit.com hooks

9e02728

for more information, see https://pre-commit.ci

headtr1ck added the topic-backends label Jul 13, 2024

headtr1ck reviewed Jul 15, 2024

View reviewed changes

xarray/backends/api.py Outdated Show resolved Hide resolved

docstring dtype naming

5762046

max-sixty merged commit 7477fd1 into pydata:main Jul 16, 2024
24 of 28 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Per-variable specification of boolean parameters in open_dataset #9218

Per-variable specification of boolean parameters in open_dataset #9218

Ostheer commented Jul 8, 2024 •

edited

Loading

Ostheer commented Jul 12, 2024

max-sixty commented Jul 12, 2024

headtr1ck commented Jul 12, 2024

Ostheer commented Jul 12, 2024

headtr1ck left a comment

Ostheer commented Jul 13, 2024

headtr1ck commented Jul 13, 2024

Ostheer commented Jul 13, 2024

headtr1ck commented Jul 13, 2024

max-sixty commented Jul 16, 2024

welcome bot commented Jul 16, 2024

Per-variable specification of boolean parameters in open_dataset #9218

Per-variable specification of boolean parameters in open_dataset #9218

Conversation

Ostheer commented Jul 8, 2024 • edited Loading

Ostheer commented Jul 12, 2024

max-sixty commented Jul 12, 2024

headtr1ck commented Jul 12, 2024

Ostheer commented Jul 12, 2024

headtr1ck left a comment

Choose a reason for hiding this comment

Ostheer commented Jul 13, 2024

headtr1ck commented Jul 13, 2024

Ostheer commented Jul 13, 2024

headtr1ck commented Jul 13, 2024

max-sixty commented Jul 16, 2024

welcome bot commented Jul 16, 2024

Ostheer commented Jul 8, 2024 •

edited

Loading