[BUG] Assigning scalar boolean to a Series w/ nulls results in wrong data type

[randerzander]

Using latest nightly cudf conda packages:

Assigning w/ booleans at DF creation time works as expected:

>>> df = cudf.DataFrame({'val': [True, None, False]})
>>> df.dtypes
val     bool
dtype: object

But assigning scalar after initializing w/ all nulls gives an unexpected float64 dtype where I'd expect a bool:

>>> df = cudf.DataFrame({'val': [None, None, None]})
>>> df.dtypes
val    float64
dtype: object

Pandas's behavior:

>>> pdf = pd.DataFrame({'val': [None, None, None]})
>>> pdf
    val
0  None
1  None
2  None
>>> pdf.dtypes
val    object
dtype: object
>>> pdf['val'] = False
>>> pdf
     val
0  False
1  False
2  False
>>> pdf.dtypes
val    bool
dtype: object

[beckernick]

From offline discussion, the following example illustrates the discrepancy in coercing (or not coercing) from the original default dtype to bool.

import cudf
import pandas as pd
​
df = cudf.DataFrame({'val': [None, None, None]})
print(df.val.dtype)
df["val"] = True
print(df)
print(df.val.dtype, "\n")
​
df = pd.DataFrame({'val': [None, None, None]})
print(df.val.dtype)
df["val"] = True
print(df)
print(df.val.dtype)

# cuDF
float64
   val
0  1.0
1  1.0
2  1.0
float64 

# pandas
object
    val
0  True
1  True
2  True
bool

[shwina]

As Nick mentioned, the crux of the problem is that we default to float as the type of an "all-nulls" column, where Pandas defaults to object:

In [8]: cudf.Series([None, None])
Out[8]:
0    <NA>
1    <NA>
dtype: float64

In [10]: pd.Series([None, None])
Out[10]:
0    None
1    None
dtype: object

We could default to object too here, and this would work as expected. Although note that objectin cuDF is simply an alias for string.

In [13]: df = cudf.DataFrame({"val": cudf.Series([None, None], dtype="object")})

In [14]: df
Out[14]:
    val
0  <NA>
1  <NA>

In [15]: df["val"] = True

In [16]: df
Out[16]:
    val
0  True
1  True

[github-actions]

This issue has been labeled inactive-30d due to no recent activity in the past 30 days. Please close this issue if no further response or action is needed. Otherwise, please respond with a comment indicating any updates or changes to the original issue and/or confirm this issue still needs to be addressed. This issue will be labeled inactive-90d if there is no activity in the next 60 days.