Make chunk manager an option in set_options
#9362
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
dcherian
approved these changes
Aug 14, 2024
TomNicholas
added
the
topic-chunked-arrays
|
Good idea! I think this should be documented and tested though. It can be documented in the paragraph you linked to, and tested using the options context manager in a simple new test. |
|
Please add a line in the docstring of |
TomNicholas
approved these changes
Aug 15, 2024
Done - thanks @TomNicholas |
dcherian
added a commit
to dcherian/xarray
that referenced
this pull request
Aug 17, 2024
* main: Improve error message for missing coordinate index (pydata#9370) Add flaky to TestNetCDF4ViaDaskData (pydata#9373) Make chunk manager an option in `set_options` (pydata#9362) Revise (pydata#9371) Remove duplicate word from docs (pydata#9367) Adding open_groups to BackendEntryPointEngine, NetCDF4BackendEntrypoint, and H5netcdfBackendEntrypoint (pydata#9243)
dcherian
added a commit
to TomNicholas/xarray
that referenced
this pull request
Aug 22, 2024
* main: (214 commits) Adds copy parameter to __array__ for numpy 2.0 (pydata#9393) `numpy 2` compatibility in the `pydap` backend (pydata#9391) pyarrow dependency added to doc environment (pydata#9394) Extend padding functionalities (pydata#9353) refactor GroupBy internals (pydata#9389) Combine `UnsignedIntegerCoder` and `CFMaskCoder` (pydata#9274) passing missing parameters to ZarrStore.open_store when opening a datatree (pydata#9377) Fix tests on big-endian systems (pydata#9380) Improve error message on `ds['x', 'y']` (pydata#9375) Improve error message for missing coordinate index (pydata#9370) Add flaky to TestNetCDF4ViaDaskData (pydata#9373) Make chunk manager an option in `set_options` (pydata#9362) Revise (pydata#9371) Remove duplicate word from docs (pydata#9367) Adding open_groups to BackendEntryPointEngine, NetCDF4BackendEntrypoint, and H5netcdfBackendEntrypoint (pydata#9243) Revise (pydata#9366) Fix rechunking to a frequency with empty bins. (pydata#9364) whats-new entry for dropping python 3.9 (pydata#9359) drop support for `python=3.9` (pydata#8937) Revise (pydata#9357) ...
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
It would be useful to have a way to choose a chunk manager if more than one is found (see https://docs.xarray.dev/en/latest/internals/chunked-arrays.html#user-interface for how it works). The use case I have is a codebase that uses Xarray and Dask (and has lots of Dask imports), and which I'd like to be able to run on Cubed too (as well as Dask). Rather than converting it all in one go, it would be easier to convert it to use Xarray-only APIs gradually. The changes involve moving
da.map_blockstoxr.apply_ufunc, for example.Using the the current PR I can have both Dask and Cubed packages installed and switch to use Cubed with
xr.set_option(chunk_manager="cubed"). This allows me to run a subset of the tests on Cubed while not having to worry about the rest of the Dask imports in the codebase.Does this look like a good way to achieve this @TomNicholas?
whats-new.rstapi.rst