Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add open_datatree benchmark #9158

Merged
merged 44 commits into from
Jul 1, 2024
Merged

Add open_datatree benchmark #9158

merged 44 commits into from
Jul 1, 2024

Conversation

aladinor
Copy link
Contributor

@aladinor aladinor commented Jun 24, 2024 •
edited by TomNicholas
Loading

aladinor and others added 30 commits May 7, 2024 14:07
@aladinor
open_datatree performance improvement on NetCDF files
14aaf56
@aladinor
fixing issue with forward slashes
3a5edb4
@aladinor
Merge branch 'main' into datatree-zarr
72d7660
@aladinor
fixing issue with pytest
d9dde29
@aladinor
fixing issue with pytest
2bc5e73
@aladinor
Merge branch 'main' into datatree-zarr
89fb4fb
@aladinor
open datatree in zarr format improvement
0343f10
@aladinor
Merge branch 'main' into datatree-zarr
93e1d59
@aladinor
fixing incompatibility in returned object
ac11b3e
@aladinor
Merge branch 'datatree-zarr' of https://github.com/aladinor/xarray in…
6d0ee13 
…to datatree-zarr

merging into same branch
@aladinor
Merge branch 'main' into datatree-zarr
91c5f0a
@aladinor
Merge branch 'main' into datatree-zarr
3363e91
@aladinor
passing group parameter to opendatatree method and reducing duplicate…
7bba52c 
…d code
@aladinor
Merge branch 'datatree-zarr' of https://github.com/aladinor/xarray in…
725aed7 
…to datatree-zarr

merging branches
@aladinor
passing group parameter to opendatatree method - NetCDF
903effd
@aladinor
Merge branch 'main' into datatree-zarr
d468478
@aladinor @TomNicholas
Update xarray/backends/netCDF4_.py
51da175 
renaming variables

Co-authored-by: Tom Nicholas <[email protected]>
@aladinor
Merge branch 'main' into datatree-zarr
24881bd
@aladinor
renaming variables
5f4bff1
@aladinor
renaming variables
41ceb4f
@aladinor
renaming group_store variable
f18ead6
@aladinor
removing _open_datatree_netcdf function not used anymore in open_data…
33d9769 
…tree implementations
@aladinor
improving performance of open_datatree method
3345b92
@aladinor
renaming 'i' variable within list comprehension in open_store method …
3cb131c 
…for zarr datatree
@aladinor
using the default generator instead of loading zarr groups in memory
6a759c0
@aladinor
fixing issue with group path to avoid using group[1:] notation. Addin…
6c00641 
…g group variable typing hints (str | Iterable[str] | callable) under the open_datatree for h5 files. Finally, separating positional from keyword args
@aladinor
fixing issue with group path to avoid using group[1:] notation and ad…
189b497 
…ding group variable typing hints (str | Iterable[str] | callable) under the open_datatree method for netCDF files
@aladinor
fixing issue with group path to avoid using group[1:] notation and ad…
a9c306d 
…ding group variable typing hints (str | Iterable[str] | callable) under the open_datatree method for zarr files
@aladinor
Merge branch 'main' into datatree-zarr
fad0e76
@aladinor
Merge branch 'main' into datatree-zarr
792f9c7
aladinor and others added 11 commits June 4, 2024 13:08
@aladinor
adding 'mode' parameter to open_datatree method
8c5796f
@aladinor
adding 'mode' parameter to H5NetCDFStore.open method
728b374
@kmuehlbauer
Merge branch 'main' into datatree-zarr
74b9a7c
@aladinor
Merge branch 'main' into datatree-zarr
e298ac4
@aladinor
Merge branch 'main' into datatree-zarr
833c978
@aladinor
adding new entry related to open_datatree performance improvement
4ff6035
@aladinor
adding new entry related to open_datatree performance improvement
3844dea
@aladinor
Getting rid of unnecessary parameters for 'open_datatree' method for …
456ce29 
…netCDF4 and Hdf5 backends
@aladinor
passing parent argument into _iter_zarr_groups instead of group[1:] f…
141fe1c 
…or creating group path
@aladinor
passing parent argument into _iter_zarr_groups instead of group[1:] f…
83ce963 
…or creating group path
@aladinor
adding benchmark test for opening a deeply nested data tree. This inc…
8a006ee 
…lude a new class named 'IONestedDataTree' and another class for benchmarck named 'IOReadDataTreeNetCDF4'
@dcherian dcherian added the run-benchmark Run the ASV benchmark workflow label Jun 24, 2024
@flamingbear flamingbear added the topic-DataTree Related to the implementation of a DataTree class label Jun 25, 2024
@owenlittlejohns owenlittlejohns mentioned this pull request Jun 26, 2024
Merged
3 tasks
dcherian
dcherian reviewed Jul 1, 2024
View reviewed changes
doc/whats-new.rst Outdated Show resolved Hide resolved
@dcherian dcherian changed the title open_datatree benchmark Add open_datatree benchmark Jul 1, 2024
@dcherian dcherian merged commit 90e4486 into pydata:main Jul 1, 2024
25 of 27 checks passed
@TomNicholas
Copy link
Member

Thanks for following up with this @aladinor !

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@dcherian dcherian dcherian left review comments

Assignees
No one assigned
Labels
run-benchmark Run the ASV benchmark workflow topic-DataTree Related to the implementation of a DataTree class
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Add benchmark test for open_datatree
5 participants
@aladinor @TomNicholas @dcherian @flamingbear @kmuehlbauer