Add mode='a-': Do not overwrite coordinates when appending to Zarr with append_dim
#8428
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
github-actions
bot
added
topic-backends
topic-zarr
This was referenced Nov 12, 2023
dcherian
force-pushed
the
fix-zarr-append
branch
2 times, most recently
from
9f51c58 to
fe36873
Compare
dcherian
force-pushed
the
fix-zarr-append
branch
from
fe36873 to
f472e9c
Compare
max-sixty
approved these changes
Nov 20, 2023
There was a problem hiding this comment.
Looks great.
One thing we could consider adding (not to this PR), is some logging on what exactly gets written. I can imagine this being a bit confusing, and in my own code I've found logging the dataset that's about to be written quite helpful. It would be the first place we add logging to xarray though...
|
What is up with the "CI / macos-latest py3.11" test? It's taking hours. Is this a known issue? |
I've had it occasionally get stuck. It'll never finish and then timeout. So don't wait for it... I'm not sure what's causing it. |
* main: [skip-ci] dev whats-new (pydata#8467) 2023.11.0 Whats-new (pydata#8461) migrate the other CI to python 3.11 (pydata#8416) preserve vlen string dtypes, allow vlen string fill_values (pydata#7869) Pin mypy < 1.7 (pydata#8458) Fix typos found by codespell (pydata#8457) [skip-ci] Small updates to IO docs. (pydata#8452) Deprecate certain cftime frequency strings following pandas (pydata#8415) Added driver parameter for h5netcdf (pydata#8360) Raise exception in to_dataset if resulting variable is also the name of a coordinate (pydata#8433) Automatic region detection and transpose for `to_zarr()` (pydata#8434) remove `cdms2` (pydata#8441) Remove PseudoNetCDF (pydata#8446) Pin pint to >=0.22 (pydata#8445) Remove keep_attrs from resample signature (pydata#8444) Rename `to_array` to `to_dataarray` (pydata#8438) Add missing DataArray.dt.total_seconds() method (pydata#8435) Declare Dataset, DataArray, Variable, GroupBy unhashable (pydata#8392)
* upstream/main: Raise an informative error message when object array has mixed types (pydata#4700) Start renaming `dims` to `dim` (pydata#8487) Reduce redundancy between namedarray and variable tests (pydata#8405) Fix Zarr region transpose (pydata#8484) Refine rolling_exp error messages (pydata#8485) Use numbagg for `ffill` by default (pydata#8389) Fix bug for categorical pandas index with categories with EA dtype (pydata#8481) Improve "variable not found" error message (pydata#8474) Add whatsnew for pydata#8475 (pydata#8478) Allow `rank` to run on dask arrays (pydata#8475) Fix mypy tests (pydata#8476) Use concise date format when plotting (pydata#8449) Fix `map_blocks` docs' formatting (pydata#8464) Consolidate `_get_alpha` func (pydata#8465)
dcherian
changed the title
append_dimappend_dim
|
Have not actually done an Xarray PR in quite a while. 😊 Thanks for your help everyone! |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This implements the 1b option described in #8427.
whats-new.rst