Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ENH: Enhancement for spss kwargs #56434

Merged
merged 51 commits into from
Aug 27, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
51 commits
Select commit Hold shift + click to select a range
dc191e3
Bug fix for spss kwargs
astronights Dec 10, 2023
c90c257
Update to whatsnew with bug fix details
astronights Dec 10, 2023
56f0840
Black formatting fix
astronights Dec 10, 2023
f9e9c91
Fix for whatsnew rst pre-commit
astronights Dec 10, 2023
0968e43
Sort whatsnew alphabetically
astronights Dec 10, 2023
397adca
Fixing alphabetical sorting to whatsnew
astronights Dec 10, 2023
f12b301
Sorting whatsnew
astronights Dec 10, 2023
df2fd08
Fixing kwargs
astronights Dec 11, 2023
68db7ba
Fixing tests
astronights Dec 11, 2023
c351e45
Merge branch 'main' into bugfix-spss-kwargs
astronights Dec 13, 2023
45d8d59
Test fixes
astronights Jan 1, 2024
d17a898
Minor change in kwargs
astronights Jan 1, 2024
4faae6f
Resolving conflict
astronights Jan 1, 2024
3eea943
Merge branch 'main' into bugfix-spss-kwargs
astronights Jan 11, 2024
02c9006
Resolving PR comments
astronights Jan 15, 2024
82cc30f
Merge branch 'bugfix-spss-kwargs' of https://github.com/astronights/p…
astronights Jan 15, 2024
ffd032e
Resolving mypy issue
astronights Jan 15, 2024
f043010
Docstring update for kwargs mypy
astronights Jan 15, 2024
24a7ee7
Merge branch 'main' into bugfix-spss-kwargs
astronights Jan 15, 2024
61e298a
Updates based on PR comments
astronights Jan 16, 2024
baa80ce
Merge branch 'bugfix-spss-kwargs' of https://github.com/astronights/p…
astronights Jan 16, 2024
47d191d
Fixing kwargs types
astronights Jan 16, 2024
00d64fc
Merge branch 'main' into bugfix-spss-kwargs
astronights Jan 16, 2024
01bb03f
Merge branch 'main' into bugfix-spss-kwargs
astronights Jan 17, 2024
8ef79d3
Finalizing updates based on suggestions
astronights Jan 19, 2024
d0b6abc
Merge branch 'bugfix-spss-kwargs' of https://github.com/astronights/p…
astronights Jan 19, 2024
d3be415
Merge branch 'main' into bugfix-spss-kwargs
astronights Jan 19, 2024
2ed0671
Updating to try to fix other random test fails
astronights Jan 19, 2024
f4c6a98
Merge branch 'bugfix-spss-kwargs' of https://github.com/astronights/p…
astronights Jan 19, 2024
83102b0
Unrelated test fix
astronights Jan 19, 2024
20ef6f5
Test fixes for dims/sizes
astronights Jan 19, 2024
2e8a88d
Merge branch 'main' into bugfix-spss-kwargs
astronights Jan 22, 2024
1371a24
Merge branch 'main' into bugfix-spss-kwargs
astronights Jan 23, 2024
0d3d87c
Merge branch 'main' into bugfix-spss-kwargs
astronights Jan 24, 2024
ab6e232
Merge branch 'main' into bugfix-spss-kwargs
astronights Jan 24, 2024
4c9246a
Merge branch 'main' into bugfix-spss-kwargs
astronights Jan 25, 2024
b2e06a6
Updates to whatsnew
astronights Jan 26, 2024
3524db0
Case fix
astronights Jan 26, 2024
9592112
Update to reword to enhancements
astronights Jan 26, 2024
cbef35e
Merge branch 'main' into bugfix-spss-kwargs
astronights Jan 26, 2024
94800db
Merge branch 'main' into bugfix-spss-kwargs
astronights Jan 27, 2024
ee08fd5
Merge branch 'main' into bugfix-spss-kwargs
astronights Jan 29, 2024
d1b0ece
Merge branch 'main' into bugfix-spss-kwargs
astronights Jan 29, 2024
186b457
Merge branch 'main' into bugfix-spss-kwargs
astronights Jan 30, 2024
c32fd1f
Merge branch 'main' into bugfix-spss-kwargs
astronights Jan 30, 2024
5718c91
Merge branch 'main' into bugfix-spss-kwargs
astronights Jan 30, 2024
ac2c2b5
Merge branch 'main' into bugfix-spss-kwargs
astronights Jan 31, 2024
a8b8637
Resolving merge conflict
astronights Feb 3, 2024
9a7d2b0
Updating to latest
astronights Mar 9, 2024
b5678f9
Merging main
astronights Aug 26, 2024
45407e6
Merge branch 'main' into bugfix-spss-kwargs
astronights Aug 27, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions doc/source/whatsnew/v3.0.0.rst
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,7 @@ Other enhancements
- :class:`pandas.api.typing.SASReader` is available for typing the output of :func:`read_sas` (:issue:`55689`)
- :func:`DataFrame.to_excel` now raises an ``UserWarning`` when the character count in a cell exceeds Excel's limitation of 32767 characters (:issue:`56954`)
- :func:`pandas.merge` now validates the ``how`` parameter input (merge type) (:issue:`59435`)
- :func:`read_spss` now supports kwargs to be passed to pyreadstat (:issue:`56356`)
- :func:`read_stata` now returns ``datetime64`` resolutions better matching those natively stored in the stata format (:issue:`55642`)
- :meth:`DataFrame.agg` called with ``axis=1`` and a ``func`` which relabels the result index now raises a ``NotImplementedError`` (:issue:`58807`).
- :meth:`Index.get_loc` now accepts also subclasses of ``tuple`` as keys (:issue:`57922`)
Expand Down
15 changes: 13 additions & 2 deletions pandas/io/spss.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,9 @@
from __future__ import annotations

from typing import TYPE_CHECKING
from typing import (
TYPE_CHECKING,
Any,
)

from pandas._libs import lib
from pandas.compat._optional import import_optional_dependency
Expand All @@ -24,6 +27,7 @@ def read_spss(
usecols: Sequence[str] | None = None,
convert_categoricals: bool = True,
dtype_backend: DtypeBackend | lib.NoDefault = lib.no_default,
**kwargs: Any,
) -> DataFrame:
"""
Load an SPSS file from the file path, returning a DataFrame.
Expand All @@ -47,6 +51,10 @@ def read_spss(
nullable :class:`ArrowDtype` :class:`DataFrame`

.. versionadded:: 2.0
**kwargs
Additional keyword arguments that can be passed to :func:`pyreadstat.read_sav`.

.. versionadded:: 3.0

Returns
-------
Expand Down Expand Up @@ -74,7 +82,10 @@ def read_spss(
usecols = list(usecols) # pyreadstat requires a list

df, metadata = pyreadstat.read_sav(
stringify_path(path), usecols=usecols, apply_value_formats=convert_categoricals
stringify_path(path),
usecols=usecols,
apply_value_formats=convert_categoricals,
**kwargs,
)
df.attrs = metadata.__dict__
if dtype_backend is not lib.no_default:
Expand Down
16 changes: 16 additions & 0 deletions pandas/tests/io/test_spss.py
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,22 @@ def test_spss_labelled_str(datapath):
tm.assert_frame_equal(df, expected)


@pytest.mark.filterwarnings("ignore::pandas.errors.ChainedAssignmentError")
@pytest.mark.filterwarnings("ignore:ChainedAssignmentError:FutureWarning")
def test_spss_kwargs(datapath):
# test file from the Haven project (https://haven.tidyverse.org/)
# Licence at LICENSES/HAVEN_LICENSE, LICENSES/HAVEN_MIT
fname = datapath("io", "data", "spss", "labelled-str.sav")

df = pd.read_spss(fname, convert_categoricals=True, row_limit=1)
expected = pd.DataFrame({"gender": ["Male"]}, dtype="category")
tm.assert_frame_equal(df, expected)

df = pd.read_spss(fname, convert_categoricals=False, row_offset=1)
expected = pd.DataFrame({"gender": ["F"]})
tm.assert_frame_equal(df, expected)


@pytest.mark.filterwarnings("ignore::pandas.errors.ChainedAssignmentError")
@pytest.mark.filterwarnings("ignore:ChainedAssignmentError:FutureWarning")
def test_spss_umlauts(datapath):
Expand Down
Loading