From 4e6c7bad35ae3886378ec50a79dac0368df875a8 Mon Sep 17 00:00:00 2001 From: Ewan Keith Date: Tue, 27 Jun 2023 13:58:06 +0100 Subject: [PATCH] Docs: Add Spark `timestamp_ntz` type to docs (#7917) * add support for spark TimestampNtz type to docs * Update type casing in spark docs Co-authored-by: Fokko Driesprong * Update type casing in spark docs Co-authored-by: Fokko Driesprong --------- Co-authored-by: Fokko Driesprong --- docs/spark-writes.md | 41 +++++++++++++++++++++-------------------- 1 file changed, 21 insertions(+), 20 deletions(-) diff --git a/docs/spark-writes.md b/docs/spark-writes.md index 5a312da7f1ee..bffbbf9be0b9 100644 --- a/docs/spark-writes.md +++ b/docs/spark-writes.md @@ -397,25 +397,26 @@ so you may want to understand the type conversion in Iceberg in prior to design This type conversion table describes how Spark types are converted to the Iceberg types. The conversion applies on both creating Iceberg table and writing to Iceberg table via Spark. -| Spark | Iceberg | Notes | -|-----------------|-------------------------|-------| -| boolean | boolean | | -| short | integer | | -| byte | integer | | -| integer | integer | | -| long | long | | -| float | float | | -| double | double | | -| date | date | | -| timestamp | timestamp with timezone | | -| char | string | | -| varchar | string | | -| string | string | | -| binary | binary | | -| decimal | decimal | | -| struct | struct | | -| array | list | | -| map | map | | +| Spark | Iceberg | Notes | +|-----------------|----------------------------|-------| +| boolean | boolean | | +| short | integer | | +| byte | integer | | +| integer | integer | | +| long | long | | +| float | float | | +| double | double | | +| date | date | | +| timestamp | timestamp with timezone | | +| timestamp_ntz | timestamp without timezone | | +| char | string | | +| varchar | string | | +| string | string | | +| binary | binary | | +| decimal | decimal | | +| struct | struct | | +| array | list | | +| map | map | | {{< hint info >}} The table is based on representing conversion during creating table. In fact, broader supports are applied on write. Here're some points on write: @@ -438,7 +439,7 @@ This type conversion table describes how Iceberg types are converted to the Spar | date | date | | | time | | Not supported | | timestamp with timezone | timestamp | | -| timestamp without timezone | | Not supported | +| timestamp without timezone | timestamp_ntz | | | string | string | | | uuid | string | | | fixed | binary | |