Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[FEA] Support Delta Lake optimized write #7393

Closed
jlowe opened this issue Dec 19, 2022 · 0 comments · Fixed by #7749
Closed

[FEA] Support Delta Lake optimized write #7393

jlowe opened this issue Dec 19, 2022 · 0 comments · Fixed by #7749
Assignees
@jlowe
Labels
feature request New feature or request performance A performance related task/issue

Comments

@jlowe
Copy link
Member

jlowe commented Dec 19, 2022

#7288 added support for Delta Lake writes, but the writes fallback to the CPU if any auto-optimization is specified on the write, either as part of the write command or via a property set on the table. Ideally the RAPIDS Accelerator should be able to support the required semantics for the auto-optimize write settings for delta.autoOptimize.optimizeWrite.
@jlowe jlowe added feature request New feature or request ? - Needs Triage Need team to review and classify labels Dec 19, 2022
@jlowe jlowe mentioned this issue Dec 19, 2022
Open
12 tasks
@sameerz sameerz added performance A performance related task/issue and removed ? - Needs Triage Need team to review and classify labels Dec 20, 2022
@jlowe jlowe self-assigned this Feb 10, 2023
@jlowe jlowe mentioned this issue Feb 13, 2023
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@jlowe jlowe

Labels
feature request New feature or request performance A performance related task/issue
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Support Delta Lake optimized write on Databricks jlowe/spark-rapids
2 participants
@jlowe @sameerz