You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Describe the bug
In CSV it is possible to have a malformed line where there is not data for each entry on a line at the end.
A,B,C
number,
In these cases Spark will insert a null no matter what, but cudf always treats it like an empty string, and then applies the rules for null values. So if the null value is an empty string, which is the default, then everything looks fine. If not then cudf produces different results.
Steps/Code to reproduce bug
We have an integration test for this test_basic_read in the CSV tests for trucks-null.csv where nullValue is set to null
The text was updated successfully, but these errors were encountered:
Describe the bug
In CSV it is possible to have a malformed line where there is not data for each entry on a line at the end.
In these cases Spark will insert a null no matter what, but cudf always treats it like an empty string, and then applies the rules for null values. So if the null value is an empty string, which is the default, then everything looks fine. If not then cudf produces different results.
Steps/Code to reproduce bug
We have an integration test for this
test_basic_read
in the CSV tests fortrucks-null.csv
wherenullValue
is set tonull
The text was updated successfully, but these errors were encountered: