Skip to content

Commit

Permalink
Docs: Add Spark timestamp_ntz type to docs (#7917)
Browse files Browse the repository at this point in the history
* add support for spark TimestampNtz type to docs

* Update type casing in spark docs

Co-authored-by: Fokko Driesprong <[email protected]>

* Update type casing in spark docs

Co-authored-by: Fokko Driesprong <[email protected]>

---------

Co-authored-by: Fokko Driesprong <[email protected]>
  • Loading branch information
Ewan Keith and Fokko authored Jun 27, 2023
1 parent 15a9af7 commit 4e6c7ba
Showing 1 changed file with 21 additions and 20 deletions.
41 changes: 21 additions & 20 deletions docs/spark-writes.md
Original file line number Diff line number Diff line change
Expand Up @@ -397,25 +397,26 @@ so you may want to understand the type conversion in Iceberg in prior to design

This type conversion table describes how Spark types are converted to the Iceberg types. The conversion applies on both creating Iceberg table and writing to Iceberg table via Spark.

| Spark | Iceberg | Notes |
|-----------------|-------------------------|-------|
| boolean | boolean | |
| short | integer | |
| byte | integer | |
| integer | integer | |
| long | long | |
| float | float | |
| double | double | |
| date | date | |
| timestamp | timestamp with timezone | |
| char | string | |
| varchar | string | |
| string | string | |
| binary | binary | |
| decimal | decimal | |
| struct | struct | |
| array | list | |
| map | map | |
| Spark | Iceberg | Notes |
|-----------------|----------------------------|-------|
| boolean | boolean | |
| short | integer | |
| byte | integer | |
| integer | integer | |
| long | long | |
| float | float | |
| double | double | |
| date | date | |
| timestamp | timestamp with timezone | |
| timestamp_ntz | timestamp without timezone | |
| char | string | |
| varchar | string | |
| string | string | |
| binary | binary | |
| decimal | decimal | |
| struct | struct | |
| array | list | |
| map | map | |

{{< hint info >}}
The table is based on representing conversion during creating table. In fact, broader supports are applied on write. Here're some points on write:
Expand All @@ -438,7 +439,7 @@ This type conversion table describes how Iceberg types are converted to the Spar
| date | date | |
| time | | Not supported |
| timestamp with timezone | timestamp | |
| timestamp without timezone | | Not supported |
| timestamp without timezone | timestamp_ntz | |
| string | string | |
| uuid | string | |
| fixed | binary | |
Expand Down

0 comments on commit 4e6c7ba

Please sign in to comment.