Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

REGR: fix bug in DatetimeIndex slicing with irregular or unsorted indices #37023

Merged
merged 24 commits into from
Oct 26, 2020
Merged

REGR: fix bug in DatetimeIndex slicing with irregular or unsorted indices #37023

Show file tree
Hide file tree
Changes from 7 commits
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
f1d4a10
remove assert in core/generic.py and add test for datetimeindex slicing
CloseChoice Oct 10, 2020
cd1a136
fix formatting in test_generic
CloseChoice Oct 10, 2020
fe478c4
fix test and remove commented line
CloseChoice Oct 10, 2020
15a9185
updated tests according to PR comments
CloseChoice Oct 10, 2020
6caba47
slice with loc to prevent future warning
CloseChoice Oct 10, 2020
5703994
revert changes in io/formats/css.py
CloseChoice Oct 10, 2020
4a7ff03
run black on test_timeseries.py
CloseChoice Oct 10, 2020
25b229c
move tests to indexing/test_partial, add whatsnew entry
CloseChoice Oct 11, 2020
9bb10fe
reformat test_timeseries.py
CloseChoice Oct 11, 2020
9d82f83
Merge branch 'master' of https://github.com/pandas-dev/pandas into BU…
CloseChoice Oct 11, 2020
968fdf4
update whatsnew
CloseChoice Oct 11, 2020
89b0248
Merge branch 'master' into BUG-regression-datetimeindex-slicing
CloseChoice Oct 16, 2020
3710514
add suggestion from comment
CloseChoice Oct 16, 2020
237b085
use result and expected in tests
CloseChoice Oct 16, 2020
43a2a26
Merge branch 'master' of https://github.com/pandas-dev/pandas into BU…
CloseChoice Oct 20, 2020
632855b
Update doc/source/whatsnew/v1.1.4.rst
CloseChoice Oct 20, 2020
e79cdac
Merge branch 'master' of https://github.com/pandas-dev/pandas into BU…
CloseChoice Oct 21, 2020
e5f3615
Merge branch 'BUG-regression-datetimeindex-slicing' of github.com:Clo…
CloseChoice Oct 21, 2020
4261f02
Merge branch 'master' of https://github.com/pandas-dev/pandas into BU…
CloseChoice Oct 22, 2020
583003e
Merge remote-tracking branch 'upstream/master' into BUG-regression-da…
simonjayhawkins Oct 26, 2020
e246520
Update pandas/tests/indexing/test_partial.py
MarcoGorelli Oct 26, 2020
9cf919e
Update pandas/tests/indexing/test_partial.py
MarcoGorelli Oct 26, 2020
7f693dc
Update pandas/tests/indexing/test_partial.py
MarcoGorelli Oct 26, 2020
02ae223
Update pandas/tests/indexing/test_partial.py
MarcoGorelli Oct 26, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion pandas/core/generic.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3736,7 +3736,6 @@ def _slice(self: FrameOrSeries, slobj: slice, axis=0) -> FrameOrSeries:

Slicing with this method is *always* positional.
"""
assert isinstance(slobj, slice), type(slobj)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is not the patch; something is going down the wrong path and ending up here. this routine is only for slices.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, I will look into that. I just wanted to note that in version 1.0.9 it went down just the same path except that the exception wasn't there yet. In this sense we are not talking about a regression but about fixing a bug which seems to have been there since a while ago.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@CloseChoice can you check if the suggestion from here #35509 (comment) passes the tests you added?

axis = self._get_block_manager_axis(axis)
result = self._constructor(self._mgr.get_slice(slobj, axis=axis))
result = result.__finalize__(self)
Expand Down
19 changes: 19 additions & 0 deletions pandas/tests/frame/test_timeseries.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -63,3 +63,22 @@ def test_datetime_assignment_with_NaT_and_diff_time_units(self):
{0: [1, None], "new": [1e9, None]}, dtype="datetime64[ns]"
)
tm.assert_frame_equal(result, expected)

def test_slice_irregular_datetime_index_with_nan(self):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

move these to pandas/tests/indexing/test_partial.py

always use
result=
expected=
tm.assert_.....

# GH36953
index = pd.to_datetime(["2012-01-01", "2012-01-02", "2012-01-03", None])
df = pd.DataFrame(range(len(index)), index=index)
expected = pd.DataFrame(range(len(index[:3])), index=index[:3])
tm.assert_frame_equal(df["2012-01-01":"2012-01-04"], expected)

def test_slice_datetime_index(self):
# GH35509
df = pd.DataFrame(
{"col1": ["a", "b", "c"], "col2": [1, 2, 3]},
index=pd.to_datetime(["2020-08-01", "2020-07-02", "2020-08-05"]),
)
expected = pd.DataFrame(
{"col1": ["a", "c"], "col2": [1, 3]},
index=pd.to_datetime(["2020-08-01", "2020-08-05"]),
)
tm.assert_frame_equal(df.loc["2020-08"], expected)