Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pickle breaking changes with minor releases / Column has no attribute default #1188

Closed
Zahlii opened this issue May 16, 2023 · 2 comments
Closed
Labels
bug Something isn't working

Comments

@Zahlii
Copy link

Zahlii commented May 16, 2023

Describe the bug

When using pickle to save schematas, there currently is no guarantee even for minor releases to be compatible. As an example, run the following file twice, once with pandas 0.14.5, once with 0.15.1.

When attempting to unpickle and apply the schema saved with 0.14.5 with 0.15.1, the following error is thrown:

  File "/opt/homebrew/Caskroom/miniforge/base/envs/flow-venv-3.9-conda/lib/python3.9/site-packages/pandera/api/pandas/container.py", line 410, in __call__
    return self.validate(
  File "/opt/homebrew/Caskroom/miniforge/base/envs/flow-venv-3.9-conda/lib/python3.9/site-packages/pandera/api/pandas/container.py", line 341, in validate
    return self._validate(
  File "/opt/homebrew/Caskroom/miniforge/base/envs/flow-venv-3.9-conda/lib/python3.9/site-packages/pandera/api/pandas/container.py", line 371, in _validate
    return self.get_backend(check_obj).validate(
  File "/opt/homebrew/Caskroom/miniforge/base/envs/flow-venv-3.9-conda/lib/python3.9/site-packages/pandera/backends/pandas/container.py", line 98, in validate
    results = check(*args)  # type: ignore [operator]
  File "/opt/homebrew/Caskroom/miniforge/base/envs/flow-venv-3.9-conda/lib/python3.9/site-packages/pandera/backends/pandas/container.py", line 146, in run_schema_component_checks
    result = schema_component.validate(
  File "/opt/homebrew/Caskroom/miniforge/base/envs/flow-venv-3.9-conda/lib/python3.9/site-packages/pandera/api/pandas/components.py", line 161, in validate
    return self.get_backend(check_obj).validate(
  File "/opt/homebrew/Caskroom/miniforge/base/envs/flow-venv-3.9-conda/lib/python3.9/site-packages/pandera/backends/pandas/components.py", line 98, in validate
    validate_column(check_obj, column_name)
  File "/opt/homebrew/Caskroom/miniforge/base/envs/flow-venv-3.9-conda/lib/python3.9/site-packages/pandera/backends/pandas/components.py", line 57, in validate_column
    super(ColumnBackend, self).validate(
  File "/opt/homebrew/Caskroom/miniforge/base/envs/flow-venv-3.9-conda/lib/python3.9/site-packages/pandera/backends/pandas/array.py", line 49, in validate
    if pd.notna(schema.default):
AttributeError: 'Column' object has no attribute 'default'
`
``

- [ x ] I have checked that this issue has not already been reported.
- [ x ] I have confirmed this bug exists on the latest version of pandera.
- [ x ] (optional) I have confirmed this bug exists on the master branch of pandera.


```python

import glob
import pickle as pkl
import pandera as pda
import pandas as pd
x = pd.DataFrame({"A": [1,2,3]})
s = pda.DataFrameSchema({"A": pda.Column(pda.Int, nullable=True)})

with open(f"schema_{pda.__version__}.pkl", "wb") as f:
    pkl.dump(s, f)


for pth in glob.glob("schema*.pkl"):
    print(pth)
    with open(pth, "rb") as f:
        s2 = pkl.load(f)
    s2(x)

Expected behavior

Script runs through with both versions.

Desktop (please complete the following information):

  • OS: MacOS
  • Version: 0.15.1
@kykyi
Copy link
Contributor

kykyi commented May 18, 2023

@Zahlii this was an attribute added in Release 0.15, I've opened a small PR to check this attribute exists before calling it. This should fix your issue!

@kykyi
Copy link
Contributor

kykyi commented Mar 4, 2024

This can be closed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

3 participants