Add support for pandas.BooleanDtype

[mattHawthorn]

I see that pandera now has support for the new extension dtypes added in pandas 1.0.0, but it appears that pandas.BooleanDtype (added primarily to support nullable boolean columns) is missing.

For example:

In [1]: import pandas as pd

In [2]: import pandera as pa

In [3]: s = pd.Series([True, False, None], dtype=pd.BooleanDtype())

In [4]: schema = pa.SeriesSchema(pa.Bool, nullable=True)

In [5]: schema.validate(s)
---------------------------------------------------------------------------
SchemaError                               Traceback (most recent call last)
<ipython-input-6-edc2d4210cf9> in <module>
----> 1 schema.validate(s)

~/anaconda3/envs/py37/lib/python3.7/site-packages/pandera-0.6.2-py3.7.egg/pandera/schemas.py in validate(self, check_obj, head, tail, sample, random_state, lazy, inplace)
   2059         try:
   2060             super().validate(
-> 2061                 check_obj, head, tail, sample, random_state, lazy, inplace
   2062             )
   2063         except errors.SchemaErrors as err:

~/anaconda3/envs/py37/lib/python3.7/site-packages/pandera-0.6.2-py3.7.egg/pandera/schemas.py in validate(self, check_obj, head, tail, sample, random_state, lazy, inplace)
   1823                     msg,
   1824                     failure_cases=scalar_failure_case(str(series_dtype)),
-> 1825                     check=f"pandas_dtype('{self.dtype}')",
   1826                 ),
   1827             )

~/anaconda3/envs/py37/lib/python3.7/site-packages/pandera-0.6.2-py3.7.egg/pandera/error_handlers.py in collect_error(self, reason_code, schema_error, original_exc)
     30         """
     31         if not self._lazy:
---> 32             raise schema_error from original_exc
     33
     34         # delete data of validated object from SchemaError object to prevent

SchemaError: expected series 'None' to have type bool, got boolean

I suspect it wouldn't be hard to add a pandera.BOOL dtype, following the same convention as was followed in reflecting pandas.Int8Dtype with pandera.INT8 and pandas.StringDtype with pandera.STRING. Is this easily doable?

[cosmicBboy]

hey @mattHawthorn! the pandera pandas_dtype also does accept any of the pandas extension types, including pd.BooleanDtype() or the uninstantiated pd.BooleanDtype (in the case of extension dtypes that don't take any arguments).

So this should work:

In [1]: import pandas as pd

In [2]: import pandera as pa
s =
In [3]: s = pd.Series([True, False, None], dtype=pd.BooleanDtype())

In [4]: schema = pa.SeriesSchema(pd.BooleanDtype(), nullable=True)

In [5]: schema(s)
Out[5]:
0     True
1    False
2     <NA>
dtype: boolean

In [6]: schema = pa.SeriesSchema(pd.BooleanDtype, nullable=True)

In [7]: schema(s)
Out[7]:
0     True
1    False
2     <NA>
dtype: boolean

[mfarragher]

@cosmicBboy I really like what I've seen of this library so far. Will pd.BooleanDtype() be supported in YAML schemas in future?

I often have datasets with dozens of columns, where there's at least one column with a nullable Pandas 'boolean' dtype. That would help with maintainability of those large schemas.

[cosmicBboy]

hi @mfarragher yes pd.BooleanDtype() will be supported in the 0.7.0 release, so that'll be coming soon!