From e8d88eee711be45f04a29ee2400b6d3a2395f7c1 Mon Sep 17 00:00:00 2001 From: Sergei Grebnov Date: Sun, 20 Oct 2024 10:13:24 -0700 Subject: [PATCH] Add Arrow to native type mapping for PostgreSQL acceleration (#546) --- .../data-accelerators/postgres/index.md | 30 +++++++++++++++++++ 1 file changed, 30 insertions(+) diff --git a/spiceaidocs/docs/components/data-accelerators/postgres/index.md b/spiceaidocs/docs/components/data-accelerators/postgres/index.md index 8c85265a..e1a33031 100644 --- a/spiceaidocs/docs/components/data-accelerators/postgres/index.md +++ b/spiceaidocs/docs/components/data-accelerators/postgres/index.md @@ -80,3 +80,33 @@ datasets: - The Postgres federated queries may result in unexpected result types due to the difference in DataFusion and Postgres size increase rules. Please explicitly specify the expected output type of aggregation functions when writing query involving Postgres table in Spice. For example, rewrite `SUM(int_col)` into `CAST (SUM(int_col) as BIGINT`. ::: + +## Arrow to PostgreSQL Type Mapping + +The table below lists the supported [Apache Arrow data types](https://arrow.apache.org/rust/arrow/datatypes/enum.DataType.html) and their mappings to [PostgreSQL types](https://www.postgresql.org/docs/current/datatype.html) when stored + +| Arrow Type | sea_query ColumnType | PostgreSQL Type | +| -------------------------------------- | ----------------------- | ----------------------------- | +| `Int8` | `TinyInteger` | `smallint` | +| `Int16` | `SmallInteger` | `smallint` | +| `Int32` | `Integer` | `integer` | +| `Int64` | `BigInteger` | `bigint` | +| `UInt8` | `TinyUnsigned` | `smallint` | +| `UInt16` | `SmallUnsigned` | `smallint` | +| `UInt32` | `Unsigned` | `bigint` | +| `UInt64` | `BigUnsigned` | `numeric` | +| `Decimal128` / `Decimal256` | `Decimal` | `decimal` | +| `Float32` | `Float` | `real` | +| `Float64` | `Double` | `double precision` | +| `Utf8 / LargeUtf8` | `Text` | `text` | +| `Boolean` | `Boolean` | `bool` | +| `Binary / LargeBinary` | `VarBinary` | `bytea` | +| `FixedSizeBinary` | `Binary` | `bytea` | +| `Timestamp` (no Timezone) | `Timestamp` | `timestamp` without time zone | +| `Timestamp` (with Timezone) | `TimestampWithTimeZone` | `timestamp` with time zone | +| `Date32` / `Date64` | `Date` | `date` | +| `Time32` / `Time64` | `Time` | `time` | +| `Interval` | `Interval` | `interval` | +| `Duration` | `BigInteger` | `bigint` | +| `List` / `LargeList` / `FixedSizeList` | `Array` | `array` | +| `Struct` | `N/A` | `Composite` (Custom type) |