Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[#1576] feat(api): add partition management API #1577

Merged
merged 10 commits into from
Jan 26, 2024

Conversation

mchades
Copy link
Contributor

@mchades mchades commented Jan 17, 2024

What changes were proposed in this pull request?

  • add partition operation APIs
  • add partition model definition
  • partition can be operated as: table.supportPartitions().xxxPartition(xx, xx)

Why are the changes needed?

Fix: #1576

Does this PR introduce any user-facing change?

no really, does not implement yet

How was this patch tested?

UTs added

import java.util.Map;

/** A partition represents a result of partitioning a table. */
public interface Partition extends Expression {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why Partitioin extens Expression?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because partitions often need to be explained by literals and field references, so I make it extends from Expression

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1,

Because partitions often need to be explained by literals and field references, so I make it extends from Expression

I do not see any requirement for it to extend Expression.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All right, it seems that we don't need it at this phase, I will remove this inheritance relationship for now.

import java.util.Map;
import java.util.Objects;

/** The helper class for partition expressions. */
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please provide more descriptions about IdentityPartition, rangePartition and listPartition, I'm confused

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

More comments are added to the static method, and UTs can also be referred to

@FANNG1 FANNG1 requested a review from qqqttt123 January 17, 2024 13:28
import java.util.Map;

/** A partition represents a result of partitioning a table. */
public interface Partition extends Expression {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1,

Because partitions often need to be explained by literals and field references, so I make it extends from Expression

I do not see any requirement for it to extend Expression.

* @return The altered partition.
* @throws NoSuchPartitionException If the partition does not exist.
*/
default Partition alterPartition(String partitionName, PartitionChange... changes)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What kind of partition changes are we planning to support now?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We currently have no plans to support this.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can remove this if we don't support it currently.

@mchades mchades self-assigned this Jan 18, 2024
@mchades
Copy link
Contributor Author

mchades commented Jan 18, 2024

@TEOTEO520 can you also help to review if you have time. :)

* @return The altered partition.
* @throws NoSuchPartitionException If the partition does not exist.
*/
default Partition alterPartition(String partitionName, PartitionChange... changes)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can remove this if we don't support it currently.

api/src/main/java/com/datastrato/gravitino/rel/Table.java Outdated Show resolved Hide resolved
@mchades mchades requested a review from jerryshao January 25, 2024 03:01
@mchades mchades requested a review from jerryshao January 25, 2024 13:54
@mchades
Copy link
Contributor Author

mchades commented Jan 25, 2024

@jerryshao All comments have been resolved. Please review again.

Partition[] listPartitions();

/**
* Get a partition by partition name
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You'd better add more examples about how to get list partition or range partition as we discussed offline.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

added

@mchades mchades requested a review from jerryshao January 25, 2024 15:20
@jerryshao jerryshao merged commit a2f1fba into apache:main Jan 26, 2024
12 checks passed
@mchades mchades deleted the manage-part branch November 22, 2024 03:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Subtask] Complete the metadata and interface design for partition management
6 participants