Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[FEA] Retry/SplitAndRetry on Parquet Writes #8028

Closed
Tracked by #8029
revans2 opened this issue Apr 4, 2023 · 1 comment · Fixed by #8243
Closed
Tracked by #8029

[FEA] Retry/SplitAndRetry on Parquet Writes #8028

revans2 opened this issue Apr 4, 2023 · 1 comment · Fixed by #8243
Assignees
@andygrove
Labels
cudf_dependency An issue or PR with this label depends on a new feature in cudf reliability Features to improve reliability or bugs that severly impact the reliability of the plugin

Comments

@revans2
Copy link
Collaborator

revans2 commented Apr 4, 2023

Is your feature request related to a problem? Please describe.
Writing to Parquet with compression and everything can be very memory intensive. We add in support for ORC, but we should also enable it for Parquet once CUDF gives us the go ahead that the chunked writer API would not cause data corruption.
@revans2 revans2 added feature request New feature or request ? - Needs Triage Need team to review and classify cudf_dependency An issue or PR with this label depends on a new feature in cudf reliability Features to improve reliability or bugs that severly impact the reliability of the plugin labels Apr 4, 2023
@revans2 revans2 mentioned this issue Apr 4, 2023
Closed
6 tasks
@mattahrens mattahrens removed the ? - Needs Triage Need team to review and classify label Apr 4, 2023
@ttnghia
Copy link
Collaborator

ttnghia commented Apr 7, 2023

Depends on:

@sameerz sameerz removed the feature request New feature or request label Apr 17, 2023
@andygrove andygrove mentioned this issue May 8, 2023
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@andygrove andygrove

Labels
cudf_dependency An issue or PR with this label depends on a new feature in cudf reliability Features to improve reliability or bugs that severly impact the reliability of the plugin
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Enable retry for Parquet writes [databricks] andygrove/spark-rapids
5 participants
@andygrove @revans2 @mattahrens @sameerz @ttnghia