BUG: Date objects cannot be compared against a DatetimeIndex

[knabben]

• I have checked that this issue has not already been reported.

• I have confirmed this bug exists on the latest version of pandas.

• (optional) I have confirmed this bug exists on the master branch of pandas.

---

Note: Please read this guide detailing how to provide the necessary information for us to reproduce your bug.

Code Sample, a copy-pastable example

Version 1.0.5

holidays = USFederalHolidayCalendar().holidays()
>>> datetime.datetime(2018, 7, 4) in holidays
True
>>> datetime.date(2018, 7, 4) in holidays
True
>>> pd.__version__
'1.0.5'

Version 1.1.0

holidays = USFederalHolidayCalendar().holidays()
>>> datetime.datetime(2018, 7, 4) in holidays
True
>>> datetime.date(2018, 7, 4) in holidays
False
>>> pd.__version__
'1.1.0'
>>>

Problem description

On version 1.0.5 it was possible to compare via contains a datetime.date object against a DatetimeIndex, this behavior is not true anymore for the 1.1.0, is this an expected behavior to allow only recognized scalars objects?

Expected Output

Allow dates to be compared with 00:00:00 time.

Output of pd.show_versions()

INSTALLED VERSIONS

commit : d9fff27
python : 3.6
OS : Darwin

pandas : 1.1.0
numpy : 1.19.1
Cython : None

[simonjayhawkins]

Thanks @knabben for the report

On version 1.0.5 it was possible to compare via contains a datetime.date object against a DatetimeIndex, this behavior is not true anymore for the 1.1.0, is this an expected behavior to allow only recognized scalars objects?

#31023 is the cause of this regression, so doesn't appear to be an intentional change cc @jbrockmendel

9b0ef5d is the first bad commit
commit 9b0ef5d
Author: jbrockmendel [email protected]
Date: Tue Jan 14 19:10:24 2020 -0800

refactor DTI.get_loc (#31023)

[jreback]

this needs more discussion

[dwardzinski-jsf]

I hope this gets more attention. This also broke DataFrame/Series indexing:

>>> date = datetime.date(2000, 1, 1)
>>> s = pd.Series([1], index=pd.to_datetime([date]))
>>> s.loc['2000-01-01']
1
>>> s.loc[date]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/dwardzinski/.local/lib/python3.8/site-packages/pandas/core/indexing.py", line 879, in __getitem__
    return self._getitem_axis(maybe_callable, axis=axis)
  File "/home/dwardzinski/.local/lib/python3.8/site-packages/pandas/core/indexing.py", line 1110, in _getitem_axis
    return self._get_label(key, axis=axis)
  File "/home/dwardzinski/.local/lib/python3.8/site-packages/pandas/core/indexing.py", line 1059, in _get_label
    return self.obj.xs(label, axis=axis)
  File "/home/dwardzinski/.local/lib/python3.8/site-packages/pandas/core/generic.py", line 3482, in xs
    loc = self.index.get_loc(key)
  File "/home/dwardzinski/.local/lib/python3.8/site-packages/pandas/core/indexes/datetimes.py", line 622, in get_loc
    raise KeyError(key)
KeyError: datetime.date(2000, 1, 1)

This broke a lot of stuff for me, since my software relies pretty heavily on indexing with date objects.
Also, #35478 (@knabben's PR) doesn't appear to fix this issue unfortunately.

[jreback]

there is not a PR to actually change this, nor has this been discussed, moving off 1.2

[jbrockmendel]

The current behavior of DatetimeIndex refusing to compare to a date object is correct, and we need to change the Timestamp behavior to match it #36131 along with the stdlib.

[simonjayhawkins]

The current behavior of DatetimeIndex refusing to compare to a date object is correct

should this behaviour be deprecated first?

and for the indexing case, #35466 (comment) (maybe need to create separate issue), is KeyError appropriate or would TypeError now be more appropriate.

to be consistent with Python list indexing...

>>> [1,2,3]["a"]
<stdin>:1: SyntaxWarning: list indices must be integers or slices, not str; perhaps you missed a comma?
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: list indices must be integers or slices, not str
>>>

perhaps something like TypeError: DatetimeIndex indices must be a time-like string, pd.Timestamp, datetime.datetime or an array of those or slices, not datetime.date

[jbrockmendel]

KeyError appropriate or would TypeError now be more appropriate.

In 1.1.0 we made it so that failed label-based lookups always raise KeyError (see _whatsnew_110.notable_bug_fixes.indexing_raises_key_errors)

[sam-s]

So, what are the plans wrt this?
This is a regression bug that seems to have been tenured into a feature.