Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Iceberg]Support metadata delete with predicate on non-identity partition columns #22554

Conversation

hantangwangd
Copy link
Member

@hantangwangd hantangwangd commented Apr 18, 2024

Description

in PR #21999, we have supported pushing down thoroughly the predicates on non-identity partition columns if they align with partitioning boundaries. So we can as well support the metadata delete on such predicates based on the same judgement logic.

Motivation and Context

Metadata-based deletion performs better than row-based deletion, and it do not corrupt the Iceberg data file metrics, so we should use it whenever possible.

Test Plan

  • Newly added test cases to show metadata-based deletion with predicates on year/month/day/hour/truncate transform columns

Contributor checklist

  • Please make sure your submission complies with our development, formatting, commit message, and attribution guidelines.
  • PR description addresses the issue accurately and concisely. If the change is non-trivial, a GitHub Issue is referenced.
  • Documented new properties (with its default value), SQL syntax, functions, or other functionality.
  • If release notes are required, they follow the release notes guidelines.
  • Adequate tests were added if applicable.
  • CI passed.

Release Notes

== RELEASE NOTES ==

Iceberg Changes
* Support metadata delete with predicate on non-identity partition columns when they align with partitioning boundaries

ZacBlanco
ZacBlanco previously approved these changes Apr 24, 2024
Copy link
Contributor

@ZacBlanco ZacBlanco left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@hantangwangd hantangwangd force-pushed the support_non_identity_for_metadata_delete branch from 34e4d6e to 43323bf Compare May 8, 2024 07:17
@hantangwangd hantangwangd force-pushed the support_non_identity_for_metadata_delete branch from 43323bf to 2eea744 Compare May 11, 2024 07:44
@hantangwangd hantangwangd force-pushed the support_non_identity_for_metadata_delete branch from 2eea744 to 6c0406a Compare May 14, 2024 17:06
@hantangwangd hantangwangd merged commit 21e301c into prestodb:master May 14, 2024
56 checks passed
@hantangwangd hantangwangd deleted the support_non_identity_for_metadata_delete branch May 14, 2024 23:08
@wanglinsong wanglinsong mentioned this pull request Jun 25, 2024
36 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants