Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Document Arrow to native type mapping for PostgreSQL acceleration #546

Merged
merged 2 commits into from
Oct 20, 2024
Merged

Document Arrow to native type mapping for PostgreSQL acceleration #546

Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
03a7927
Add Arrow to PostgreSQL Type Mapping
sgrebnov Oct 15, 2024
cf4a74a
Update index.md
sgrebnov Oct 17, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 30 additions & 0 deletions spiceaidocs/docs/components/data-accelerators/postgres/index.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -80,3 +80,33 @@ datasets:
- The Postgres federated queries may result in unexpected result types due to the difference in DataFusion and Postgres size increase rules. Please explicitly specify the expected output type of aggregation functions when writing query involving Postgres table in Spice. For example, rewrite `SUM(int_col)` into `CAST (SUM(int_col) as BIGINT`.

:::

## Arrow to PostgreSQL Type Mapping

The table below lists the supported [Apache Arrow data types](https://arrow.apache.org/rust/arrow/datatypes/enum.DataType.html) and their mappings to [PostgreSQL types](https://www.postgresql.org/docs/current/datatype.html) when stored

| Arrow Type | sea_query ColumnType | PostgreSQL Type |
| ------------------------------ | -------------------------- | ---------------------------- |
| Int8 | TinyInteger | smallint |
| Int16 | SmallInteger | smallint |
| Int32 | Integer | integer |
| Int64 | BigInteger | bigint |
| UInt8 | TinyUnsigned | smallint |
| UInt16 | SmallUnsigned | smallint |
| UInt32 | Unsigned | bigint |
| UInt64 | BigUnsigned | numeric |
| Decimal128 / Decimal256 | Decimal | decimal |
| Float32 | Float | real |
| Float64 | Double | double precision |
| Utf8 / LargeUtf8 | Text | text |
| Boolean | Boolean | bool |
| Binary / LargeBinary | VarBinary | bytea |
| FixedSizeBinary | Binary | bytea |
| Timestamp (no Timezone) | Timestamp | timestamp without time zone |
| Timestamp (with Timezone) | TimestampWithTimeZone | timestamp with time zone |
| Date32 / Date64 | Date | date |
| Time32 / Time64 | Time | time |
| Interval | Interval | interval |
| Duration | BigInteger | bigint |
| List / LargeList / FixedSizeList | Array | array |
| Struct | N/A | Composite (Custom type) |
sgrebnov marked this conversation as resolved.
Show resolved Hide resolved