tests for arrays with units

[keewis]

This PR adds tests that verify that xarray properly wraps arrays with units.

• Passes black . && mypy . && flake8
• Fully documented, including whats-new.rst for all changes
  and api.rst for new API

They are currently restricted to using pint but may easily be extended to
use other unit libraries. However, most of the libraries I know use
subclassing instead of wrapping, so using them together with dask might
not be ideal as unit conflicts will only be detected on compute().

As a reference for myself, here are the operations I plan on adding tests for:

• top-level functions:
  • replication: full_like, ones_like, zeros_like
• dataarray methods
  • creating: DataArray()
  • representation: __repr__, __str__
  • content manipulation: assign_coords, assign_attrs, pipe, rename, swap_dims, expand_dims, drop, reset_coords, copy, astype, item
  • indexing: loc, isel, sel, squeeze, interp, interp_like, reindex, reindex_like
  • missing value handling: isnull, notnull, combine_first, count, dropna, fillna, ffill, bfill, interpolate_na, where, isin
  • comparisons: equals, identical, broadcast_equals
  • aggregation: all, any, argmax, argmin, max, mean, median, min, prod, sum, std, var
  • ndarray methods: argsort, clip, conj, conjugate, imag, searchsorted, round, real, T, cumsum, cumprod, rank
  • reshaping and reorganizing: transpose, stack, unstack, set_index, reset_index, reorder_levels, shift, roll, sortby, to_unstacked_dataset
  • arithmetic: unary (-, abs, round), binary (+, *, @)
  • ufuncs (univariate: sin, cos, bivariate: minimum, maximum)
  • grouped operations: assign_coords, first, last, fillna, where, quantile
  • computation: reduce, groupby, groupby_bins, rolling, rolling_exp, coarsen, resample, diff, dot, quantile, differentiate, integrate

The parts of the API that are currently not covered by this PR are:

• top level:
  • combining: align, concat, merge, auto_combine, combine_by_coords, combine_nested
  • computation: broadcast, dot, apply_ufunc
• dataarray:
  • computation: get_axis_num
• dataset

From what is already done, mypy . fails because of pint (this the same as with quantities in #2956). What would I do to fix that?

[max-sixty]

Great! This is going to be exciting. Ping here for any questions!

From what is already done, mypy . fails because of pint (this the same as with quantities in #2956). What would I do to fix that?

V easy - add to setup.cfg like other libraries listed there

[jthielen]

@keewis: In case it helps, I've added a bunch of additional __array_function__ implementations in pint in andrewgsavage/pint#6, which will hopefully get merged soon into the main hgrecco/pint#764 PR. If through these tests with xarray you find something missing on pint's end, let me know and I'd be glad to help.

[keewis]

@jthielen: thanks, I will look at these when I have the time.

Could you take a look at the failing test for the DataArray-constructor? It fails because pint emits a UnitStrippedWarning on calling hasattr(data, "__array_function__"), which I believe should not happen (correct me if I'm wrong, though).

[jthielen]

I noticed you have @pytest.mark.filterwarnings("error::pint.errors.UnitStrippedWarning"), which I believe is what is raising that warning to the error level. What happens when you remove that?

As to why the warning is happening in the first place, I think that is because the __array_function__ implementation in pint right now only exists in a PR. So, when xarray looks for an __array_function__ attribute on the pint Quantity right now, it falls back to this:

https://github.com/hgrecco/pint/blob/2afdc4bf5c5727ed9cef6fdaccb00b88813c4a24/pint/quantity.py#L1438-L1449

Hence, the UnitStrippedWarning. However, hasattr(data, "__array_function__") should still work, albeit for the wrong reason since pint doesn't yet have a true __array_function__ method.

[keewis]

ah, okay, that makes sense. I will leave the filterwarning since I believe the warning should not be raised, but mark that test as xfail until a released pint version implements __array_function__.

[crusaderky]

@keewis looks good. Also something equivalent in installing.rst on the line that mentions pint, please

[keewis]

I don't really know what to write about

readers will be much more interested in what they can actually do with pint

because a feature list seems to be a bit too long to me.

re installing.rst: is adding (experimental) enough, or should the line with pip also appear there?

[crusaderky]

whatsnew.rst:

- Added integration tests against pint <https://pint.readthedocs.io/>_.

  .. note::

    At the moment of writing, these tests *as well as the ability to use pint in general*
    require `a highly experimental version of pint
    <https://github.com/andrewgsavage/pint/pull/6>`_ (install with 
    ``pip install git+https://github.com/andrewgsavage/pint.git@refs/pull/6/head)``.
    Even with it, interaction with non-numpy array libraries, e.g. dask or sparse, is broken.

installing.rst

- `pint <https://pint.readthedocs.io/>`_: for units of measure.

  .. note::

    At the moment of writing, xarray requires a `highly experimental version of pint 
    <https://github.com/andrewgsavage/pint/pull/6>`_ (install with
    ``pip install git+https://github.com/andrewgsavage/pint.git@refs/pull/6/head)``.
    Even with it, interaction with non-numpy array libraries, e.g. dask or sparse, is broken.

[crusaderky]

Also please move the what's new blob from 0.14.0 to 0.14.1

[keewis]

done

[max-sixty]

I think these tests are unrelated and we can merge?

=================================== FAILURES ===================================
_________________________ test_datetime_reduce[False] __________________________

dask = False

    @arm_xfail
    @pytest.mark.parametrize("dask", [False, True])
    def test_datetime_reduce(dask):
        time = np.array(pd.date_range("15/12/1999", periods=11))
        time[8:11] = np.nan
        da = DataArray(np.linspace(0, 365, num=11), dims="time", coords={"time": time})
    
        if dask and has_dask:
            chunks = {"time": 5}
            da = da.chunk(chunks)
    
        actual = da["time"].mean()
>       assert not pd.isnull(actual)
E       AssertionError: assert not True
E        +  where True = <function isna at 0x7fcbdefdbae8>(<xarray.DataArray 'time' ()>\narray('NaT', dtype='datetime64[ns]'))
E        +    where <function isna at 0x7fcbdefdbae8> = pd.isnull

xarray/tests/test_duck_array_ops.py:288: AssertionError
__________________________ test_datetime_reduce[True] __________________________

dask = True

    @arm_xfail
    @pytest.mark.parametrize("dask", [False, True])
    def test_datetime_reduce(dask):
        time = np.array(pd.date_range("15/12/1999", periods=11))
        time[8:11] = np.nan
        da = DataArray(np.linspace(0, 365, num=11), dims="time", coords={"time": time})
    
        if dask and has_dask:
            chunks = {"time": 5}
            da = da.chunk(chunks)
    
        actual = da["time"].mean()
>       assert not pd.isnull(actual)
E       AssertionError: assert not True
E        +  where True = <function isna at 0x7fcbdefdbae8>(<xarray.DataArray 'time' ()>\narray('NaT', dtype='datetime64[ns]'))
E        +    where <function isna at 0x7fcbdefdbae8> = pd.isnull

[keewis]

I would think so, too. Go ahead.

Edit: seems like other PRs have the same problem, see #3409

[max-sixty]

Great! Quite an effort @keewis —thank you!

[dcherian]

Phew thanks a lot @keewis!

[shoyer]

awesome work @keewis !