Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

objects remain unserializable after reset_index #8628

Closed
5 tasks done
bjarketol opened this issue Jan 19, 2024 · 1 comment · Fixed by #8672
Closed
5 tasks done

objects remain unserializable after reset_index #8628

bjarketol opened this issue Jan 19, 2024 · 1 comment · Fixed by #8672

Comments

@bjarketol
Copy link

What happened?

With the 2024.1 release, I am unable to write objects to netCDF after having stacked dimensions with .stack() and called .reset_index() to get rid of the multi-index

What did you expect to happen?

No response

Minimal Complete Verifiable Example

import numpy as np 
import xarray as xr 
da = xr.DataArray(np.zeros([2, 3]), dims=["x", "y"])
da = da.stack(point=("x", "y"))
da = da.reset_index("point")
da.to_netcdf("test.nc")

MVCE confirmation

  • Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue in xarray.
  • Complete example — the example is self-contained, including all data and the text of any traceback.
  • Verifiable example — the example copy & pastes into an IPython prompt or Binder notebook, returning the result.
  • New issue — a search of GitHub Issues suggests this is not a duplicate.
  • Recent environment — the issue occurs with the latest version of xarray and its dependencies.

Relevant log output

86 def ensure_not_multiindex(var: Variable, name: T_Name = None) -> None:
     87     if isinstance(var._data, indexing.PandasMultiIndexingAdapter):
---> 88         raise NotImplementedError(
     89             f"variable {name!r} is a MultiIndex, which cannot yet be "
     90             "serialized. Instead, either use reset_index() "
     91             "to convert MultiIndex levels into coordinate variables instead "
     92             "or use https://cf-xarray.readthedocs.io/en/latest/coding.html."
     93         )

NotImplementedError: variable 'x' is a MultiIndex, which cannot yet be serialized. Instead, either use reset_index() to convert MultiIndex levels into coordinate variables instead or use https://cf-xarray.readthedocs.io/en/latest/coding.html.

Anything else we need to know?

Creating the stacked object from scratch and saving it to netCDF works fine. The difference is that type(da.x.variable._data) is xarray.core.indexing.PandasMultiIndexingAdapter if it was stacked and reset and numpy.ndarray if it's created from scratch

Environment

INSTALLED VERSIONS

commit: None
python: 3.11.7 | packaged by conda-forge | (main, Dec 23 2023, 14:43:09) [GCC 12.3.0]
python-bits: 64
OS: Linux
OS-release: 5.15.133.1-microsoft-standard-WSL2
machine: x86_64
processor: x86_64
byteorder: little
LC_ALL: None
LANG: C.UTF-8
LOCALE: ('en_US', 'UTF-8')
libhdf5: 1.14.3
libnetcdf: 4.9.2

xarray: 2024.1.0
pandas: 2.1.4
numpy: 1.26.3
scipy: 1.11.4
netCDF4: 1.6.5
pydap: None
h5netcdf: 1.2.0
h5py: 3.10.0
Nio: None
zarr: 2.16.1
cftime: 1.6.3
nc_time_axis: None
iris: None
bottleneck: 1.3.7
dask: 2024.1.0
distributed: None
matplotlib: 3.8.2
cartopy: 0.22.0
seaborn: 0.13.1
numbagg: None
fsspec: 2023.12.2
cupy: None
pint: 0.23
sparse: None
flox: None
numpy_groupies: None
setuptools: 69.0.3
pip: 23.3.2
conda: None
pytest: 7.4.4
mypy: None
IPython: 8.20.0

@bjarketol bjarketol added bug needs triage Issue that has not been reviewed by xarray team member labels Jan 19, 2024
@TomNicholas TomNicholas added topic-indexing and removed needs triage Issue that has not been reviewed by xarray team member labels Jan 19, 2024
@benbovy
Copy link
Member

benbovy commented Jan 26, 2024

Oops, my mistake in #8565 (comment).

Thanks for the report @bjarketol!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants