Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

groupby_bins incorrect correspondence with labels #7766

Closed
4 tasks done
gsieros opened this issue Apr 19, 2023 · 1 comment · Fixed by #7769
Closed
4 tasks done

groupby_bins incorrect correspondence with labels #7766

gsieros opened this issue Apr 19, 2023 · 1 comment · Fixed by #7769
Labels
bug needs triage Issue that has not been reviewed by xarray team member

Comments

@gsieros
Copy link

gsieros commented Apr 19, 2023

What happened?

As of version 2023.4.0 groupby_bins stopped giving correct results. This was reported in #7751. The fix applied in 2023.4.1 changes the behaviour but does not fix the problem in my use case where strings are used as labels for the groups. It appears that the sorting that was implemented for the fix sorts based on name (so e.g. 'one' comes after 'four', see example) instead of on the lower edge. I could be wrong about the reason but accessing groups like data[group['label']] breaks

What did you expect to happen?

Until 2023.3.0 accessing groups by name gave the correct subset from the DataArray

Minimal Complete Verifiable Example

import numpy as np
import xarray as xr
import pandas as pd
import matplotlib.pyplot as plt
import sys

print(f"numpy version: {np.__version__}")
print(f"xarray version: {xr.__version__}")
print(f"pandas version: {pd.__version__}")
print(f"python version: {sys.version}")


# Generate random data 
# Make the coordiantes follow a normal distribution
np.random.seed(42)
coords = np.random.normal(5, 5, 1000)
bins = np.logspace(-4, 1, 10)
labels = ['one', 'two', 'three', 'four', 'five', 'six', 'seven',
          'eight', 'nine']
# xArray
# Make a mock dataarray
darr = xr.DataArray(coords, coords=[coords], dims=["coords"])
groups_xr = darr.groupby_bins("coords", bins, labels=labels).groups
print(groups_xr.keys())
for lab in labels:
    print(lab, end=' ')
    try:
        print(int(darr[groups_xr[lab]].count()))
    except:
        print("nan")

MVCE confirmation

  • Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue in xarray.
  • Complete example — the example is self-contained, including all data and the text of any traceback.
  • Verifiable example — the example copy & pastes into an IPython prompt or Binder notebook, returning the result.
  • New issue — a search of GitHub Issues suggests this is not a duplicate.

Relevant log output

No response

Anything else we need to know?

Output from 2023.3.0 (keys in random order, but correct access by name)

numpy version: 1.23.5
xarray version: 2023.3.0
pandas version: 1.5.3
python version: 3.10.6 (main, Mar 10 2023, 10:55:28) [GCC 11.3.0]
dict_keys(['nine', 'eight', 'seven', 'five', 'six', 'four'])
one nan
two nan
three nan
four 1
five 2
six 9
seven 27
eight 153
nine 506

Output from 2023.4.1 - Keys alphabetically sorted, wrong access by name
numpy version: 1.23.5
xarray version: 2023.4.1
pandas version: 1.5.3
python version: 3.10.6 (main, Mar 10 2023, 10:55:28) [GCC 11.3.0]
dict_keys(['eight', 'five', 'four', 'nine', 'seven', 'six'])
one nan
two nan
three nan
four 9
five 2
six 506
seven 153
eight 1
nine 27

Environment

INSTALLED VERSIONS

commit: None
python: 3.10.6 (main, Mar 10 2023, 10:55:28) [GCC 11.3.0]
python-bits: 64
OS: Linux
OS-release: 5.19.0-40-generic
machine: x86_64
processor: x86_64
byteorder: little
LC_ALL: en_US.UTF-8
LANG: el_GR.UTF-8
LOCALE: ('en_US', 'UTF-8')
libhdf5: 1.10.7
libnetcdf: 4.8.1

xarray: 2023.4.1
pandas: 1.5.3
numpy: 1.23.5
scipy: 1.10.2.dev0+2411.f2ee108
netCDF4: 1.5.8
pydap: None
h5netcdf: 1.1.0
h5py: 3.6.0
Nio: None
zarr: 2.14.2
cftime: 1.5.2
nc_time_axis: None
PseudoNetCDF: None
iris: None
bottleneck: 1.3.7
dask: 2023.4.0
distributed: 2023.4.0
matplotlib: 3.7.1
cartopy: 0.21.1
seaborn: 0.12.2
numbagg: None
fsspec: 2023.4.0
cupy: None
pint: None
sparse: None
flox: None
numpy_groupies: None
setuptools: 67.6.1
pip: 23.1
conda: None
pytest: 6.2.5
mypy: 0.942
IPython: 8.12.0
sphinx: 4.3.2

@gsieros gsieros added bug needs triage Issue that has not been reviewed by xarray team member labels Apr 19, 2023
@welcome
Copy link

welcome bot commented Apr 19, 2023

Thanks for opening your first issue here at xarray! Be sure to follow the issue template!
If you have an idea for a solution, we would really welcome a Pull Request with proposed changes.
See the Contributing Guide for more.
It may take us a while to respond here, but we really value your contribution. Contributors like you help make xarray better.
Thank you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug needs triage Issue that has not been reviewed by xarray team member
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant