Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[#2266] fix(partition): enable preassign partition when creating range and list partitioning table #3189

Merged
merged 5 commits into from
May 13, 2024

Conversation

mchades
Copy link
Contributor

@mchades mchades commented Apr 25, 2024

What changes were proposed in this pull request?

enable pre-assign partition when creating range and list partitioning table

Why are the changes needed?

Fix: #2266

Does this PR introduce any user-facing change?

yes, enable pre-assign partition when creating range and list partitioning table

How was this patch tested?

tests modified

@mchades mchades self-assigned this Apr 25, 2024
@jerryshao jerryshao requested a review from yuqi1129 May 9, 2024 12:45
@yuqi1129
Copy link
Contributor

@mchades
Can you tell me how you plan to use the pre-assigned partitioning in creating table?

@mchades
Copy link
Contributor Author

mchades commented May 10, 2024

@mchades
Copy link
Contributor Author

mchades commented May 10, 2024

@FANNG1 @jerryshao Please help to review when you have time, thanks!

@mchades mchades requested review from jerryshao and FANNG1 May 10, 2024 10:15
Copy link
Contributor

@jerryshao jerryshao left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How would specific catalog use it. I see you only do this in the interface layer, how would the catalog leverage this?

default Expression[] assignments() {
return Expression.EMPTY_EXPRESSION;
default Partition[] assignments() {
return EMPTY_PARTITIONS;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the reason of this change?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because the previous API confused Partitioning (same as Transform) with Partition, which is a wrong implementation, the assignments should be partitions, so here is the Partition instead of Expression to fix the mistake.

.withLower(LiteralDTO.NULL)
.build();
Partitioning rangePart =
RangePartitioningDTO.of(field1, new RangePartitionDTO[] {p20230101, p20230102});
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you please also add some failure tests for your change in serde.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, can you please add a mock test both in server and client side to see if the serde works well.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

added

@mchades
Copy link
Contributor Author

mchades commented May 11, 2024

How would specific catalog use it. I see you only do this in the interface layer, how would the catalog leverage this?

yes, this PR only did the API refactor, if catalog(like Doris) has the requirement, it can add the pre-assigned partition when creating the table, I think this can be done in another PR

@mchades mchades requested a review from jerryshao May 11, 2024 08:40
@jerryshao jerryshao merged commit 55626fc into apache:main May 13, 2024
22 checks passed
github-actions bot pushed a commit that referenced this pull request May 13, 2024
…e and list partitioning table (#3189)

### What changes were proposed in this pull request?

enable pre-assign partition when creating range and list partitioning
table

### Why are the changes needed?

Fix: #2266 

### Does this PR introduce _any_ user-facing change?

yes, enable pre-assign partition when creating range and list
partitioning table

### How was this patch tested?

tests modified
diqiu50 pushed a commit to diqiu50/gravitino that referenced this pull request Jun 13, 2024
…g range and list partitioning table (apache#3189)

### What changes were proposed in this pull request?

enable pre-assign partition when creating range and list partitioning
table

### Why are the changes needed?

Fix: apache#2266 

### Does this PR introduce _any_ user-facing change?

yes, enable pre-assign partition when creating range and list
partitioning table

### How was this patch tested?

tests modified
@mchades mchades deleted the preassign-partition branch November 22, 2024 03:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Subtask] Support preassigned partitions when creating partitioned table
3 participants