Skip to content

Commit

Permalink
[apache#3140] docs(partition): add docs and OAS for drop partition op…
Browse files Browse the repository at this point in the history
…eration (apache#3141)

### What changes were proposed in this pull request?

add docs and OAS for drop partition operation

### Why are the changes needed?

Fix: apache#3140

### Does this PR introduce _any_ user-facing change?

no

### How was this patch tested?

no
  • Loading branch information
mchades authored and diqiu50 committed Jun 13, 2024
1 parent 464d7b7 commit 8041b48
Show file tree
Hide file tree
Showing 3 changed files with 83 additions and 7 deletions.
54 changes: 47 additions & 7 deletions docs/manage-table-partition-using-gravitino.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,13 +19,13 @@ Although many catalogs inherently manage partitions automatically, there are sce

The following table shows the partition operations supported across various catalogs in Gravitino:

| Operation | Hive catalog | Iceberg catalog | Jdbc-Mysql catalog | Jdbc-PostgreSQL catalog |
|-----------------------|-------------------------------------------------------------------------------|-------------------------------------------------------------------------------|--------------------|-------------------------|
| Add Partition | ✔ | ✘ | ✘ | ✘ |
| Get Partition by Name | ✔ | ✘ | ✘ | ✘ |
| List Partition Names | ✔ | ✘ | ✘ | ✘ |
| List Partitions | ✔ | ✘ | ✘ | ✘ |
| Drop Partition | 🚀([Coming Soon](https://github.com/datastrato/gravitino/issues/1655)) | 🚀([Coming Soon](https://github.com/datastrato/gravitino/issues/1655)) | ✘ | ✘ |
| Operation | Hive catalog | Iceberg catalog | Jdbc-Mysql catalog | Jdbc-PostgreSQL catalog |
|-----------------------|--------------|-------------------------------------------------------------------------------|--------------------|-------------------------|
| Add Partition | ✔ | ✘ | ✘ | ✘ |
| Get Partition by Name | ✔ | ✘ | ✘ | ✘ |
| List Partition Names | ✔ | ✘ | ✘ | ✘ |
| List Partitions | ✔ | ✘ | ✘ | ✘ |
| Drop Partition | ✔ | 🚀([Coming Soon](https://github.com/datastrato/gravitino/issues/1655)) | ✘ | ✘ |

:::tip[WELCOME FEEDBACK]
If you need additional partition management support for a specific catalog, please feel free to [create an issue](https://github.com/datastrato/gravitino/issues/new/choose) on the [Gravitino repository](https://github.com/datastrato/gravitino).
Expand Down Expand Up @@ -374,5 +374,45 @@ Partition[] partitions =
.listPartitions();
```

</TabItem>
</Tabs>

### Drop a partition by name

You can drop a partition by its name via sending a `DELETE` request to the `/api/metalakes/{metalake_name}/catalogs/{catalog_name}/schemas/{schema_name}/tables/{partitioned_table_name}/partitions/{partition_name}` endpoint or by using the Gravitino Java client.
The following is an example of dropping a partition by its name:

<Tabs>
<TabItem value="shell" label="Shell">

```shell
curl -X DELETE -H "Accept: application/vnd.gravitino.v1+json" \
-H "Content-Type: application/json" \
http://localhost:8090/api/metalakes/metalake/catalogs/catalog/schemas/schema/tables/table/partitions/p20200321
```

:::tip
If the partition name contains special characters, you should use [URL encoding](https://en.wikipedia.org/wiki/Percent-encoding#Reserved_characters). For example, if the partition name is `dt=2008-08-08/country=us` you should use `dt%3D2008-08-08%2Fcountry%3Dus` in the URL.
:::

</TabItem>
<TabItem value="java" label="Java">

```java
GravitinoClient gravitinoClient = GravitinoClient
.builder("http://127.0.0.1:8090")
.withMetalake("metalake")
.build();

// Assume that you have a partitioned table named "metalake.catalog.schema.table".
Partition Partition =
gravitinoClient
.loadCatalog(NameIdentifier.of("metalake", "catalog"))
.asTableCatalog()
.loadTable(NameIdentifier.of("metalake", "catalog", "schema", "table"))
.supportPartitions()
.dropPartition("partition_name");
```

</TabItem>
</Tabs>
25 changes: 25 additions & 0 deletions docs/open-api/partitions.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -105,6 +105,22 @@ paths:
"5xx":
$ref: "./openapi.yaml#/components/responses/ServerErrorResponse"

delete:
tags:
- partition
summary: Drop partition by name
operationId: dropPartition
description: Drops the specified partition
parameters:
- $ref: "#/components/parameters/purge"
responses:
"200":
$ref: "./openapi.yaml#/components/responses/DropResponse"
"400":
$ref: "./openapi.yaml#/components/responses/BadRequestErrorResponse"
"5xx":
$ref: "./openapi.yaml#/components/responses/ServerErrorResponse"


components:
parameters:
Expand All @@ -117,6 +133,15 @@ components:
type: boolean
default: false

purge:
name: purge
in: query
description: Purge the partition data
required: false
schema:
type: boolean
default: false

partition:
name: partition
in: path
Expand Down
11 changes: 11 additions & 0 deletions docs/open-api/tables.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -141,6 +141,8 @@ paths:
- table
summary: Drop table
operationId: dropTable
parameters:
- $ref: "#/components/parameters/purge"
responses:
"200":
$ref: "./openapi.yaml#/components/responses/DropResponse"
Expand All @@ -151,6 +153,15 @@ paths:


components:
parameters:
purge:
name: purge
in: query
description: Removes both the metadata and the directory associated with the table completely and skipping trash.
required: false
schema:
type: boolean
default: false

schemas:
TableCreateRequest:
Expand Down

0 comments on commit 8041b48

Please sign in to comment.