You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Describe the bug
When a column is set to non-nullable and a nullable value is supplied in the data frame then no further validations are executed for that column and all failure cases are not reported for that column. I am hoping you'll tell me that I am doing something wrong 😃
I have checked that this issue has not already been reported.
I have confirmed this bug exists on the latest version of pandera.
(optional) I have confirmed this bug exists on the master branch of pandera.
Code Sample
importpandasaspdimportpanderaaspacolumn=pa.Column(pa.String, name="id", nullable=False, required=True, allow_duplicates=False)
column.checks.append(pa.Check.str_matches(r"^ID[\d]{3}$"))
#column.has_subcomponents = True - should i switch this on?schema=pa.DataFrameSchema({
"id": column
})
data= ["ID001", None, "XXX"]
df=pd.DataFrame(data, columns=["id"])
try:
schema.validate(df, lazy=True)
exceptpa.errors.SchemaErrorsasexc:
print(exc.failure_cases)
Output
schema_context
column
check
check_number
failure_case
index
0
Column
id
non_nullable
None
None
1
Expected behavior
I would expect all validations to be performed and failure cases to be reported, in this case i would expect the output to be
schema_context
column
check
check_number
failure_case
index
0
Column
id
non_nullable
None
None
1
1
Column
id
str_matches(...)
...
XXX
2
Desktop (please complete the following information):
OS: Amazon Linux 2
CPE OS Name: cpe:2.3:o:amazon:amazon_linux:2
Kernel: Linux 4.14.232-177.418.amzn2.x86_64
Version 0.6.4
Additional context
I have done some digging and have found this issue and #528 that offers a fix. This doesn't resolve the nullable issue (i don't think).
In schemas.py the nullable check is here and the error handler is initialized here it appears that the lazy parameter that is passed in here only comes as None unless has_subcomponents is true which has been removed in the latest dev branch.
The text was updated successfully, but these errors were encountered:
Describe the bug
When a column is set to non-nullable and a nullable value is supplied in the data frame then no further validations are executed for that column and all failure cases are not reported for that column. I am hoping you'll tell me that I am doing something wrong 😃
Code Sample
Output
Expected behavior
I would expect all validations to be performed and failure cases to be reported, in this case i would expect the output to be
Desktop (please complete the following information):
Additional context
I have done some digging and have found this issue and #528 that offers a fix. This doesn't resolve the nullable issue (i don't think).
In
schemas.py
the nullable check is here and the error handler is initialized here it appears that thelazy
parameter that is passed in here only comes asNone
unlesshas_subcomponents
is true which has been removed in the latest dev branch.The text was updated successfully, but these errors were encountered: