diff --git a/docs/apache-hive-catalog.md b/docs/apache-hive-catalog.md index 0defa234508..c4e5985d7fd 100644 --- a/docs/apache-hive-catalog.md +++ b/docs/apache-hive-catalog.md @@ -116,6 +116,10 @@ The following table lists the data types mapped from the Hive catalog to Graviti | `struct` | `struct` | 0.2.0 | | `uniontype` | `uniontype` | 0.2.0 | +:::info +Since 0.5.0, the data types other than listed above are mapped to Gravitino **[Unparsed Type](./manage-metadata-using-gravitino.md#unparsed-type)** that represents an unresolvable data type from the Hive catalog. +::: + ### Table properties Table properties supply or set metadata for the underlying Hive tables. diff --git a/docs/jdbc-mysql-catalog.md b/docs/jdbc-mysql-catalog.md index 3e46d343ea2..3a6d59f4cd5 100644 --- a/docs/jdbc-mysql-catalog.md +++ b/docs/jdbc-mysql-catalog.md @@ -103,6 +103,7 @@ Please refer to [Manage Metadata Using Gravitino](./manage-metadata-using-gravit :::info MySQL doesn't support Gravitino `Boolean` `Fixed` `Struct` `List` `Map` `Timestamp_tz` `IntervalDay` `IntervalYear` `Union` `UUID` type. +Meanwhile, the data types other than listed above are mapped to Gravitino **[Unparsed Type](./manage-metadata-using-gravitino.md#unparsed-type)** that represents an unresolvable data type since 0.5.0. ::: #### Table column auto-increment diff --git a/docs/jdbc-postgresql-catalog.md b/docs/jdbc-postgresql-catalog.md index 64c96be59f7..e3aeed73abf 100644 --- a/docs/jdbc-postgresql-catalog.md +++ b/docs/jdbc-postgresql-catalog.md @@ -108,6 +108,7 @@ Please refer to [Manage Metadata Using Gravitino](./manage-metadata-using-gravit :::info PostgreSQL doesn't support Gravitino `Fixed` `Struct` `List` `Map` `IntervalDay` `IntervalYear` `Union` `UUID` type. +Meanwhile, the data types other than listed above are mapped to Gravitino **[Unparsed Type](./manage-metadata-using-gravitino.md#unparsed-type)** that represents an unresolvable data type since 0.5.0. ::: #### Table column auto-increment diff --git a/docs/lakehouse-iceberg-catalog.md b/docs/lakehouse-iceberg-catalog.md index 22189fba3b9..bdaf7c4539e 100644 --- a/docs/lakehouse-iceberg-catalog.md +++ b/docs/lakehouse-iceberg-catalog.md @@ -215,6 +215,7 @@ Apache Iceberg doesn't support Gravitino `EvenDistribution` type. :::info Apache Iceberg doesn't support Gravitino `Varchar` `Fixedchar` `Byte` `Short` `Union` type. +Meanwhile, the data types other than listed above are mapped to Gravitino **[Unparsed Type](./manage-metadata-using-gravitino.md#unparsed-type)** that represents an unresolvable data type since 0.5.0. ::: ### Table properties diff --git a/docs/manage-metadata-using-gravitino.md b/docs/manage-metadata-using-gravitino.md index a0b5d2508f8..88c12b7a4d6 100644 --- a/docs/manage-metadata-using-gravitino.md +++ b/docs/manage-metadata-using-gravitino.md @@ -609,8 +609,8 @@ You can create a table by sending a `POST` request to the `/api/metalakes/{metal ```shell curl -X POST -H "Accept: application/vnd.gravitino.v1+json" \ -H "Content-Type: application/json" -d '{ - "name": "exmaple_table", - "comment": "This is an exmaple table", + "name": "example_table", + "comment": "This is an example table", "columns": [ { "name": "id", @@ -756,7 +756,7 @@ Map tablePropertiesMap = ImmutableMap.builder() .build(); tableCatalog.createTable( - NameIdentifier.of("metalake", "catalog", "schema", "exmaple_table"), + NameIdentifier.of("metalake", "catalog", "schema", "example_table"), new Column[] { Column.of("id", Types.IntegerType.get(), "id column comment", false, true, Literals.integerLiteral(-1)), Column.of("name", Types.VarCharType.of(500), "name column comment", true, false, Literals.NULL), @@ -768,7 +768,7 @@ tableCatalog.createTable( ), "info column comment", true, false, null), Column.of("dt", Types.DateType.get(), "dt column comment", true, false, null) }, - "This is an exmaple table", + "This is an example table", tablePropertiesMap, new Transform[] {Transforms.identity("id")}, Distributions.of(Strategy.HASH, 32, NamedReference.field("id")), @@ -823,6 +823,32 @@ The following types that Gravitino supports: The related java doc is [here](pathname:///docs/0.4.0/api/java/com/datastrato/gravitino/rel/types/Type.html). +##### Unparsed type + +Unparsed type is a special type of column type, currently serves exclusively for presenting the data type of a column when it's unsolvable. +The following shows the data structure of an unparsed type in JSON and Java, enabling easy retrieval of its value. + + + + +```json +{ + "type": "unparsed", + "unparsedType": "user-defined" +} +``` + + + + +```java +// The result of the following type is a string "user-defined" +String unparsedValue = ((UnparsedType) type).unparsedType(); +``` + + + + #### Table column default value When defining a table column, you can specify a [literal](./expression.md#literal) or an [expression](./expression.md) as the default value. The default value typically applies to new rows that are inserted into the table by the underlying catalog. @@ -906,7 +932,8 @@ tableCatalog.loadTable(NameIdentifier.of("metalake", "hive_catalog", "schema", " :::note -When Gravitino loads a table from a catalog that supports default value, if Gravitino is unable to parse the default value, it will use an **[Unparsed Expression](./expression.md#unparsed-expression)** to preserve the original default value, ensuring that the table can be loaded successfully. +- When Gravitino loads a table from a catalog with various data types, if Gravitino is unable to parse the data type, it will use an **[Unparsed Type](#unparsed-type)** to preserve the original data type, ensuring that the table can be loaded successfully. +- When Gravitino loads a table from a catalog that supports default value, if Gravitino is unable to parse the default value, it will use an **[Unparsed Expression](./expression.md#unparsed-expression)** to preserve the original default value, ensuring that the table can be loaded successfully. ::: ### Alter a table diff --git a/docs/open-api/datatype.yaml b/docs/open-api/datatype.yaml index c1afca9a357..ba5fbc2354a 100644 --- a/docs/open-api/datatype.yaml +++ b/docs/open-api/datatype.yaml @@ -14,6 +14,7 @@ components: - $ref: "#/components/schemas/ListType" - $ref: "#/components/schemas/MapType" - $ref: "#/components/schemas/UnionType" + - $ref: "#/components/schemas/UnparsedType" PrimitiveType: type: string @@ -162,4 +163,24 @@ components: comment: type: string description: The comment of the struct field - nullable: true \ No newline at end of file + nullable: true + + UnparsedType: + type: object + required: + - type + - unparsedType + properties: + type: + type: string + enum: + - "unparsed" + unparsedType: + type: string + description: The unparsed type + example: { + "type": "unparsed", + "unparsedType": [ + "user-defined" + ] + } \ No newline at end of file