Skip to content

Commit

Permalink
BUG: all not ignoring na when skipna=True (pandas-dev#55367)
Browse files Browse the repository at this point in the history
* BUG: all not ignoring na when skipna=True

* Update v2.1.2.rst

(cherry picked from commit 364c9cb)
  • Loading branch information
phofl committed Oct 4, 2023
1 parent f07deef commit 0ffef50
Show file tree
Hide file tree
Showing 3 changed files with 8 additions and 4 deletions.
1 change: 1 addition & 0 deletions doc/source/whatsnew/v2.1.2.rst
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@ Bug fixes
- Fixed bug in :meth:`DataFrame.idxmin` and :meth:`DataFrame.idxmax` raising for arrow dtypes (:issue:`55368`)
- Fixed bug in :meth:`DataFrame.interpolate` raising incorrect error message (:issue:`55347`)
- Fixed bug in :meth:`Index.insert` raising when inserting ``None`` into :class:`Index` with ``dtype="string[pyarrow_numpy]"`` (:issue:`55365`)
- Fixed bug in :meth:`Series.all` and :meth:`Series.any` not treating missing values correctly for ``dtype="string[pyarrow_numpy]"`` (:issue:`55367`)
- Silence ``Period[B]`` warnings introduced by :issue:`53446` during normal plotting activity (:issue:`55138`)
-

Expand Down
8 changes: 5 additions & 3 deletions pandas/core/arrays/string_arrow.py
Original file line number Diff line number Diff line change
Expand Up @@ -571,9 +571,11 @@ def _reduce(
self, name: str, *, skipna: bool = True, keepdims: bool = False, **kwargs
):
if name in ["any", "all"]:
arr = pc.and_kleene(
pc.invert(pc.is_null(self._pa_array)), pc.not_equal(self._pa_array, "")
)
if not skipna and name == "all":
nas = pc.invert(pc.is_null(self._pa_array))
arr = pc.and_kleene(nas, pc.not_equal(self._pa_array, ""))
else:
arr = pc.not_equal(self._pa_array, "")
return ArrowExtensionArray(arr)._reduce(
name, skipna=skipna, keepdims=keepdims, **kwargs
)
Expand Down
3 changes: 2 additions & 1 deletion pandas/tests/reductions/test_reductions.py
Original file line number Diff line number Diff line change
Expand Up @@ -1087,7 +1087,8 @@ def test_any_all_pyarrow_string(self):

ser = Series([None, "a"], dtype="string[pyarrow_numpy]")
assert ser.any()
assert not ser.all()
assert ser.all()
assert not ser.all(skipna=False)

ser = Series([None, ""], dtype="string[pyarrow_numpy]")
assert not ser.any()
Expand Down

0 comments on commit 0ffef50

Please sign in to comment.