Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[FEA] Chunked ORC reading #7131

Closed
revans2 opened this issue Nov 21, 2022 · 1 comment · Fixed by #10723
Closed

[FEA] Chunked ORC reading #7131

revans2 opened this issue Nov 21, 2022 · 1 comment · Fixed by #10723
Assignees
@ttnghia
Labels
cudf_dependency An issue or PR with this label depends on a new feature in cudf reliability Features to improve reliability or bugs that severly impact the reliability of the plugin

Comments

@revans2
Copy link
Collaborator

revans2 commented Nov 21, 2022

Is your feature request related to a problem? Please describe.
#4968 was originally filed for us to find a solution to parquet and ORC reading where we could decode the data in smaller pieces. This ended up needing CUDF changes to actually work properly. An implementation was put in place for parquet, but we also need one for ORC.
@revans2 revans2 added feature request New feature or request ? - Needs Triage Need team to review and classify cudf_dependency An issue or PR with this label depends on a new feature in cudf labels Nov 21, 2022
@sameerz sameerz added reliability Features to improve reliability or bugs that severly impact the reliability of the plugin and removed feature request New feature or request ? - Needs Triage Need team to review and classify labels Nov 22, 2022
@revans2
Copy link
Collaborator Author

revans2 commented Nov 22, 2022

the CUDF issue is rapidsai/cudf#12228

@ttnghia ttnghia mentioned this issue Apr 22, 2024
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@ttnghia ttnghia

Labels
cudf_dependency An issue or PR with this label depends on a new feature in cudf reliability Features to improve reliability or bugs that severly impact the reliability of the plugin
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Implement chunked ORC reader ttnghia/spark-rapids
3 participants
@revans2 @sameerz @ttnghia