[BUG] Orc writes don't fully support Booleans with nulls #11736
Labels
bug
Something isn't working
cudf_dependency
An issue or PR with this label depends on a new feature in cudf
Describe the bug
Customer reported a job failing with the error:
Caused by: ai.rapids.cudf.CudfException: CUDF failure at: /home/jenkins/agent/workspace/jenkins-spark-rapids-jni-release-39-cuda11/thirdparty/cudf/cpp/src/io/orc/[writer_impl.cu](http://writer_impl.cu/):940: There's currently a bug in encoding boolean columns. Suggested workaround is to convert to int8 type. Please see https://github.com/rapidsai/cudf/issues/6763 for more information.
as the error suggests it points to rapidsai/cudf#6763, which states that cudf doesn't support writing booleans with nulls that don't align on 8 bit boundaries. It requires writing at least 2 row groups where the first rows group has doesn't fully align with 8 bits and leaves unused bits.
The text was updated successfully, but these errors were encountered: