-
-
Notifications
You must be signed in to change notification settings - Fork 18.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
BUG: MultiIndex constructed with nulls inconsistent with dtypes
and get_level_values
#44792
Comments
It's not exclusive to In [7]: pd.MultiIndex.from_tuples([(1, None), (None, None), (2, 3)]).dtypes
Out[7]:
level_0 int64
level_1 int64
dtype: object
In [8]: pd.MultiIndex.from_tuples([(1, None), (None, None), (2, 3)]).get_level_values(0)
Out[8]: Float64Index([1.0, nan, 2.0], dtype='float64')
In [9]: pd.MultiIndex.from_tuples([(1, None), (None, None), (2, 3)]).get_level_values(1)
Out[9]: Float64Index([nan, nan, 3.0], dtype='float64') |
from_arrays
inconsistent with dtypesdtypes
and get_level_values
This is somehow expected, since the MultiIndex is working with codes (-1 signaling a missing value) and the NaNs are not applied to the Index itself. When extracting a single level, this is converted to an Index class, which does not support this. Maybe we can document this better |
@phofl Thanks for clarify. I guess the right place to document this is |
Yep sounds reasonable, would you be interested in submitting a pr? |
Sure! |
I have checked that this issue has not already been reported.
I have confirmed this bug exists on the latest version of pandas.
I have confirmed this bug exists on the master branch of pandas.
Reproducible Example
Issue Description
dtypes
reports that both levels are of integer types, but the returned level values are float types.Expected Behavior
Either
dtypes
reports float types ordtypes
reports nullable integer types andget_level_values
returns that.Installed Versions
INSTALLED VERSIONS
commit : 945c9ed
python : 3.8.12.final.0
python-bits : 64
OS : Linux
OS-release : 5.11.0-40-generic
Version : #44~20.04.2-Ubuntu SMP Tue Oct 26 18:07:44 UTC 2021
machine : x86_64
processor : x86_64
byteorder : little
LC_ALL : None
LANG : None
LOCALE : en_US.UTF-8
pandas : 1.3.4
numpy : 1.20.3
pytz : 2021.3
dateutil : 2.8.2
pip : 21.3.1
setuptools : 59.4.0
Cython : 0.29.24
pytest : 6.2.5
hypothesis : 6.29.0
sphinx : 4.3.1
blosc : None
feather : None
xlsxwriter : None
lxml.etree : 4.6.4
html5lib : None
pymysql : None
psycopg2 : None
jinja2 : 3.0.3
IPython : 7.30.0
pandas_datareader: None
bs4 : 4.10.0
bottleneck : None
fsspec : 2021.11.1
fastparquet : None
gcsfs : None
matplotlib : None
numexpr : None
odfpy : None
openpyxl : None
pandas_gbq : None
pyarrow : 5.0.0
pyxlsb : None
s3fs : None
scipy : None
sqlalchemy : None
tables : None
tabulate : None
xarray : None
xlrd : None
xlwt : None
numba : 0.54.1
The text was updated successfully, but these errors were encountered: