Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BUG] Parquet with RLE encoded booleans loads corrupted data #8630

Closed
jlowe opened this issue Jun 28, 2023 · 2 comments · Fixed by rapidsai/cudf#13707 or #8829
Closed

[BUG] Parquet with RLE encoded booleans loads corrupted data #8630

jlowe opened this issue Jun 28, 2023 · 2 comments · Fixed by rapidsai/cudf#13707 or #8829
Labels
bug Something isn't working cudf_dependency An issue or PR with this label depends on a new feature in cudf

Comments

@jlowe
Copy link
Member

jlowe commented Jun 28, 2023

Describe the bug
The RAPIDS Accelerator fails to properly load a Parquet file with RLE encoded booleans. Some of the boolean values should be null, but none of the loaded values are null.

Steps/Code to reproduce bug
Load the rle_boolean_encoding.parquet file from parquet-testing

Expected behavior
The data should match what is loaded from the same file via Spark CPU.
@jlowe jlowe added bug Something isn't working ? - Needs Triage Need team to review and classify labels Jun 28, 2023
@jlowe
Copy link
Member Author

jlowe commented Jul 3, 2023

Filed rapidsai/cudf#13655

@jlowe jlowe added the cudf_dependency An issue or PR with this label depends on a new feature in cudf label Jul 3, 2023
@mattahrens mattahrens removed the ? - Needs Triage Need team to review and classify label Jul 5, 2023
@sameerz sameerz linked a pull request Jul 26, 2023 that will close this issue
Fix reading of RLE encoded boolean data from parquet files with V2 page headers rapidsai/cudf#13707
Merged
3 tasks
@jlowe jlowe mentioned this issue Jul 27, 2023
Merged
@jlowe
Copy link
Member Author

jlowe commented Jul 27, 2023

@mattahrens this bug should not have been closed. Even though cudf fix has been merged, this also tracks re-enabling the test for it.

@jlowe jlowe reopened this Jul 27, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working cudf_dependency An issue or PR with this label depends on a new feature in cudf
Projects
None yet
Milestone
No milestone
2 participants
@jlowe @mattahrens