Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

use nested_duck_arrays to detect the innermost layer of duck arrays #9212

Draft
wants to merge 6 commits into
base: main
Choose a base branch
from

Conversation

keewis
Copy link
Collaborator

@keewis keewis commented Jul 7, 2024

As described in #7721 (comment), this uses nested_duck_arrays.first_layer (or should that be called innermost_layer? Not sure) to detect cupy under one or more layers of duck arrays.

This is highly experimental, and before merging this, I'd like to:

  • settle on the API in nested_duck_arrays – the name of the protocol, and the names of the functions in nested_duck_arrays are not set, yet.
  • in general: get feedback on the idea
  • get pint and dask (the most common duck arrays that create layers) to implement the protocol
  • publish nested_duck_arrays to PyPI and conda-forge

Also, the tests make use of cupy (which we don't install in CI), which means we can't really check that this works... but at least we can verify that the changes here don't affect anything else.

@keewis keewis marked this pull request as draft July 7, 2024 19:16
@TomNicholas TomNicholas added the topic-arrays related to flexible array support label Jul 8, 2024
@yt87
Copy link

yt87 commented Jul 11, 2024

Since there is already a special handling of cupy in as_shared_type, why not to add import nested_duck_arrays.dask in the try block near the top of duck_array_ops.py? That import is neccessary to handle chunked cupy arrays. That would make cupy-xarray work and, I think, it would close #9212. When dask implements the protocol, the import can be removed.

My bad. That import obviously can be done in the user code.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
topic-arrays related to flexible array support
Projects
None yet
Development

Successfully merging this pull request may close these issues.

as_shared_dtype converts scalars to 0d numpy arrays if chunked cupy is involved
3 participants