Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ENH: Enhancement for spss kwargs #56434

Merged
merged 51 commits into from
Aug 27, 2024
Merged
Show file tree
Hide file tree
Changes from 14 commits
Commits
Show all changes
51 commits
Select commit Hold shift + click to select a range
dc191e3
Bug fix for spss kwargs
astronights Dec 10, 2023
c90c257
Update to whatsnew with bug fix details
astronights Dec 10, 2023
56f0840
Black formatting fix
astronights Dec 10, 2023
f9e9c91
Fix for whatsnew rst pre-commit
astronights Dec 10, 2023
0968e43
Sort whatsnew alphabetically
astronights Dec 10, 2023
397adca
Fixing alphabetical sorting to whatsnew
astronights Dec 10, 2023
f12b301
Sorting whatsnew
astronights Dec 10, 2023
df2fd08
Fixing kwargs
astronights Dec 11, 2023
68db7ba
Fixing tests
astronights Dec 11, 2023
c351e45
Merge branch 'main' into bugfix-spss-kwargs
astronights Dec 13, 2023
45d8d59
Test fixes
astronights Jan 1, 2024
d17a898
Minor change in kwargs
astronights Jan 1, 2024
4faae6f
Resolving conflict
astronights Jan 1, 2024
3eea943
Merge branch 'main' into bugfix-spss-kwargs
astronights Jan 11, 2024
02c9006
Resolving PR comments
astronights Jan 15, 2024
82cc30f
Merge branch 'bugfix-spss-kwargs' of https://github.com/astronights/p…
astronights Jan 15, 2024
ffd032e
Resolving mypy issue
astronights Jan 15, 2024
f043010
Docstring update for kwargs mypy
astronights Jan 15, 2024
24a7ee7
Merge branch 'main' into bugfix-spss-kwargs
astronights Jan 15, 2024
61e298a
Updates based on PR comments
astronights Jan 16, 2024
baa80ce
Merge branch 'bugfix-spss-kwargs' of https://github.com/astronights/p…
astronights Jan 16, 2024
47d191d
Fixing kwargs types
astronights Jan 16, 2024
00d64fc
Merge branch 'main' into bugfix-spss-kwargs
astronights Jan 16, 2024
01bb03f
Merge branch 'main' into bugfix-spss-kwargs
astronights Jan 17, 2024
8ef79d3
Finalizing updates based on suggestions
astronights Jan 19, 2024
d0b6abc
Merge branch 'bugfix-spss-kwargs' of https://github.com/astronights/p…
astronights Jan 19, 2024
d3be415
Merge branch 'main' into bugfix-spss-kwargs
astronights Jan 19, 2024
2ed0671
Updating to try to fix other random test fails
astronights Jan 19, 2024
f4c6a98
Merge branch 'bugfix-spss-kwargs' of https://github.com/astronights/p…
astronights Jan 19, 2024
83102b0
Unrelated test fix
astronights Jan 19, 2024
20ef6f5
Test fixes for dims/sizes
astronights Jan 19, 2024
2e8a88d
Merge branch 'main' into bugfix-spss-kwargs
astronights Jan 22, 2024
1371a24
Merge branch 'main' into bugfix-spss-kwargs
astronights Jan 23, 2024
0d3d87c
Merge branch 'main' into bugfix-spss-kwargs
astronights Jan 24, 2024
ab6e232
Merge branch 'main' into bugfix-spss-kwargs
astronights Jan 24, 2024
4c9246a
Merge branch 'main' into bugfix-spss-kwargs
astronights Jan 25, 2024
b2e06a6
Updates to whatsnew
astronights Jan 26, 2024
3524db0
Case fix
astronights Jan 26, 2024
9592112
Update to reword to enhancements
astronights Jan 26, 2024
cbef35e
Merge branch 'main' into bugfix-spss-kwargs
astronights Jan 26, 2024
94800db
Merge branch 'main' into bugfix-spss-kwargs
astronights Jan 27, 2024
ee08fd5
Merge branch 'main' into bugfix-spss-kwargs
astronights Jan 29, 2024
d1b0ece
Merge branch 'main' into bugfix-spss-kwargs
astronights Jan 29, 2024
186b457
Merge branch 'main' into bugfix-spss-kwargs
astronights Jan 30, 2024
c32fd1f
Merge branch 'main' into bugfix-spss-kwargs
astronights Jan 30, 2024
5718c91
Merge branch 'main' into bugfix-spss-kwargs
astronights Jan 30, 2024
ac2c2b5
Merge branch 'main' into bugfix-spss-kwargs
astronights Jan 31, 2024
a8b8637
Resolving merge conflict
astronights Feb 3, 2024
9a7d2b0
Updating to latest
astronights Mar 9, 2024
b5678f9
Merging main
astronights Aug 26, 2024
45407e6
Merge branch 'main' into bugfix-spss-kwargs
astronights Aug 27, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion doc/source/whatsnew/v2.2.0.rst
Original file line number Diff line number Diff line change
Expand Up @@ -871,7 +871,8 @@ I/O
- Bug in :func:`read_csv` with ``engine="pyarrow"`` where ``usecols`` wasn't working with a CSV with no headers (:issue:`54459`)
- Bug in :func:`read_excel`, with ``engine="xlrd"`` (``xls`` files) erroring when the file contains ``NaN`` or ``Inf`` (:issue:`54564`)
- Bug in :func:`read_json` not handling dtype conversion properly if ``infer_string`` is set (:issue:`56195`)
- Bug in :meth:`DataFrame.to_excel`, with ``OdsWriter`` (``ods`` files) writing Boolean/string value (:issue:`54994`)
- Bug in :func:`read_spss` where kwargs were not passed to pyreadstat (:issue:`56356`)
astronights marked this conversation as resolved.
Show resolved Hide resolved
- Bug in :meth:`DataFrame.to_excel`, with ``OdsWriter`` (``ods`` files) writing boolean/string value (:issue:`54994`)
astronights marked this conversation as resolved.
Show resolved Hide resolved
- Bug in :meth:`DataFrame.to_hdf` and :func:`read_hdf` with ``datetime64`` dtypes with non-nanosecond resolution failing to round-trip correctly (:issue:`55622`)
- Bug in :meth:`DataFrame.to_stata` raising for extension dtypes (:issue:`54671`)
- Bug in :meth:`~pandas.read_excel` with ``engine="odf"`` (``ods`` files) when a string cell contains an annotation (:issue:`55200`)
Expand Down
8 changes: 7 additions & 1 deletion pandas/io/spss.py
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ def read_spss(
usecols: Sequence[str] | None = None,
convert_categoricals: bool = True,
dtype_backend: DtypeBackend | lib.NoDefault = lib.no_default,
**kwargs: dict,
astronights marked this conversation as resolved.
Show resolved Hide resolved
) -> DataFrame:
"""
Load an SPSS file from the file path, returning a DataFrame.
Expand All @@ -46,6 +47,8 @@ def read_spss(
DataFrame.

.. versionadded:: 2.0
kwargs : dict, optional
astronights marked this conversation as resolved.
Show resolved Hide resolved
Additional keyword arguments passed to :func:`pyreadstat.read_sav`.
astronights marked this conversation as resolved.
Show resolved Hide resolved

Returns
-------
Expand All @@ -64,7 +67,10 @@ def read_spss(
usecols = list(usecols) # pyreadstat requires a list

df, metadata = pyreadstat.read_sav(
stringify_path(path), usecols=usecols, apply_value_formats=convert_categoricals
stringify_path(path),
usecols=usecols,
apply_value_formats=convert_categoricals,
**kwargs,
)
df.attrs = metadata.__dict__
if dtype_backend is not lib.no_default:
Expand Down
17 changes: 17 additions & 0 deletions pandas/tests/io/test_spss.py
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,23 @@ def test_spss_labelled_str(datapath):
tm.assert_frame_equal(df, expected)


@pytest.mark.filterwarnings("ignore::pandas.errors.ChainedAssignmentError")
@pytest.mark.filterwarnings("ignore:ChainedAssignmentError:FutureWarning")
def test_spss_kwargs(datapath):
# test file from the Haven project (https://haven.tidyverse.org/)
# Licence at LICENSES/HAVEN_LICENSE, LICENSES/HAVEN_MIT
fname = datapath("io", "data", "spss", "labelled-str.sav")

df = pd.read_spss(fname, convert_categoricals=True, row_limit=1)
expected = pd.DataFrame({"gender": ["Male"]})
expected["gender"] = pd.Categorical(expected["gender"])
astronights marked this conversation as resolved.
Show resolved Hide resolved
tm.assert_frame_equal(df, expected)

df = pd.read_spss(fname, convert_categoricals=False, row_offset=1)
expected = pd.DataFrame({"gender": ["F"]})
tm.assert_frame_equal(df, expected)


@pytest.mark.filterwarnings("ignore::pandas.errors.ChainedAssignmentError")
@pytest.mark.filterwarnings("ignore:ChainedAssignmentError:FutureWarning")
def test_spss_umlauts(datapath):
Expand Down
Loading