Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[#3140] docs(partition): add docs and OAS for drop partition operation #3145

Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
54 changes: 47 additions & 7 deletions docs/manage-table-partition-using-gravitino.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,13 +19,13 @@ Although many catalogs inherently manage partitions automatically, there are sce

The following table shows the partition operations supported across various catalogs in Gravitino:

| Operation | Hive catalog | Iceberg catalog | Jdbc-Mysql catalog | Jdbc-PostgreSQL catalog |
|-----------------------|-------------------------------------------------------------------------------|-------------------------------------------------------------------------------|--------------------|-------------------------|
| Add Partition | ✔ | ✘ | ✘ | ✘ |
| Get Partition by Name | ✔ | ✘ | ✘ | ✘ |
| List Partition Names | ✔ | ✘ | ✘ | ✘ |
| List Partitions | ✔ | ✘ | ✘ | ✘ |
| Drop Partition | 🚀([Coming Soon](https://github.com/datastrato/gravitino/issues/1655)) | 🚀([Coming Soon](https://github.com/datastrato/gravitino/issues/1655)) | ✘ | ✘ |
| Operation | Hive catalog | Iceberg catalog | Jdbc-Mysql catalog | Jdbc-PostgreSQL catalog |
|-----------------------|--------------|-------------------------------------------------------------------------------|--------------------|-------------------------|
| Add Partition | ✔ | ✘ | ✘ | ✘ |
| Get Partition by Name | ✔ | ✘ | ✘ | ✘ |
| List Partition Names | ✔ | ✘ | ✘ | ✘ |
| List Partitions | ✔ | ✘ | ✘ | ✘ |
| Drop Partition | ✔ | 🚀([Coming Soon](https://github.com/datastrato/gravitino/issues/1655)) | ✘ | ✘ |

:::tip[WELCOME FEEDBACK]
If you need additional partition management support for a specific catalog, please feel free to [create an issue](https://github.com/datastrato/gravitino/issues/new/choose) on the [Gravitino repository](https://github.com/datastrato/gravitino).
Expand Down Expand Up @@ -374,5 +374,45 @@ Partition[] partitions =
.listPartitions();
```

</TabItem>
</Tabs>

### Drop a partition by name

You can drop a partition by its name via sending a `DELETE` request to the `/api/metalakes/{metalake_name}/catalogs/{catalog_name}/schemas/{schema_name}/tables/{partitioned_table_name}/partitions/{partition_name}` endpoint or by using the Gravitino Java client.
The following is an example of dropping a partition by its name:

<Tabs>
<TabItem value="shell" label="Shell">

```shell
curl -X DELETE -H "Accept: application/vnd.gravitino.v1+json" \
-H "Content-Type: application/json" \
http://localhost:8090/api/metalakes/metalake/catalogs/catalog/schemas/schema/tables/table/partitions/p20200321
```

:::tip
If the partition name contains special characters, you should use [URL encoding](https://en.wikipedia.org/wiki/Percent-encoding#Reserved_characters). For example, if the partition name is `dt=2008-08-08/country=us` you should use `dt%3D2008-08-08%2Fcountry%3Dus` in the URL.
:::

</TabItem>
<TabItem value="java" label="Java">

```java
GravitinoClient gravitinoClient = GravitinoClient
.builder("http://127.0.0.1:8090")
.withMetalake("metalake")
.build();

// Assume that you have a partitioned table named "metalake.catalog.schema.table".
Partition Partition =
gravitinoClient
.loadCatalog(NameIdentifier.of("metalake", "catalog"))
.asTableCatalog()
.loadTable(NameIdentifier.of("metalake", "catalog", "schema", "table"))
.supportPartitions()
.dropPartition("partition_name");
```

</TabItem>
</Tabs>
25 changes: 25 additions & 0 deletions docs/open-api/partitions.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -105,6 +105,22 @@ paths:
"5xx":
$ref: "./openapi.yaml#/components/responses/ServerErrorResponse"

delete:
tags:
- partition
summary: Drop partition by name
operationId: dropPartition
description: Drops the specified partition
parameters:
- $ref: "#/components/parameters/purge"
responses:
"200":
$ref: "./openapi.yaml#/components/responses/DropResponse"
"400":
$ref: "./openapi.yaml#/components/responses/BadRequestErrorResponse"
"5xx":
$ref: "./openapi.yaml#/components/responses/ServerErrorResponse"


components:
parameters:
Expand All @@ -117,6 +133,15 @@ components:
type: boolean
default: false

purge:
name: purge
in: query
description: Purge the partition data
required: false
schema:
type: boolean
default: false

partition:
name: partition
in: path
Expand Down
11 changes: 11 additions & 0 deletions docs/open-api/tables.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -141,6 +141,8 @@ paths:
- table
summary: Drop table
operationId: dropTable
parameters:
- $ref: "#/components/parameters/purge"
responses:
"200":
$ref: "./openapi.yaml#/components/responses/DropResponse"
Expand All @@ -151,6 +153,15 @@ paths:


components:
parameters:
purge:
name: purge
in: query
description: Removes both the metadata and the directory associated with the table completely and skipping trash.
required: false
schema:
type: boolean
default: false

schemas:
TableCreateRequest:
Expand Down
Loading