Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pa.dataframe_check causes generated data to be invalid #524

Closed
ghost opened this issue Jun 24, 2021 · 2 comments
Closed

pa.dataframe_check causes generated data to be invalid #524

ghost opened this issue Jun 24, 2021 · 2 comments
Labels
question Further information is requested

Comments

@ghost
Copy link

ghost commented Jun 24, 2021

Hi, below is a minimal example of the question / possible bug. Is it expected that after adding a dataframe_check in SchemaWithDFCheck that the schema no longer generates valid data according to the field column checks? This example is still somewhat non-deterministic, please let me know if there is a better way than @seed(10) to get reproducible results.

I do get a warning when running this example, but I would have expected the generated data to still be valid.

UserWarning: Dataframe check doesn't have a defined strategy. Falling back to filtering drawn values based on the check definition. This can considerably slow down data-generation.

Versions:

  • python==3.9
  • pandas==1.2.5
  • pandera==0.6.4
import pandera as pa
import pandas as pd
from pandera.typing import Series
from hypothesis import seed


class Schema(pa.SchemaModel):
    field: Series[float] = pa.Field(gt=0)


class SchemaWithDFCheck(Schema):
    @pa.dataframe_check
    def non_empty(self, df: pd.DataFrame) -> bool:
        return not df.empty


@seed(10)
def test():
    print(Schema.example(size=1))
    '''
    >>> field
    0   4.940656e-324
    '''
    print(SchemaWithDFCheck.example(size=1))
    '''
    >>> field
    0   0.0
    '''


if __name__ == '__main__':
    test()
@ghost ghost added the question Further information is requested label Jun 24, 2021
@cosmicBboy
Copy link
Collaborator

this is certainly a bug, thanks for catching @bphillips-exos! looking into it

@cosmicBboy
Copy link
Collaborator

fixed by #550 @bphillips-exos this should be out in the next 0.6.5 release!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
question Further information is requested
Projects
None yet
Development

No branches or pull requests

1 participant