From 0b24fd7807cace2f574143154a08946403dd56e6 Mon Sep 17 00:00:00 2001 From: peasee <98815791+peasee@users.noreply.github.com> Date: Wed, 13 Nov 2024 13:13:38 +1000 Subject: [PATCH 1/3] docs: Clarify DuckDB type limitations --- .../docs/components/data-accelerators/duckdb.md | 12 ++++-------- .../docs/components/data-connectors/duckdb.md | 13 +++++++++++++ 2 files changed, 17 insertions(+), 8 deletions(-) diff --git a/spiceaidocs/docs/components/data-accelerators/duckdb.md b/spiceaidocs/docs/components/data-accelerators/duckdb.md index 093d0e2f..e99dbf13 100644 --- a/spiceaidocs/docs/components/data-accelerators/duckdb.md +++ b/spiceaidocs/docs/components/data-accelerators/duckdb.md @@ -36,18 +36,14 @@ datasets: :::warning[Limitations] -- The DuckDB accelerator does not support schemas with [field types](https://duckdb.org/docs/sql/data_types/overview): nested arrays/lists, UTF8/string arrays/lists, boolean arrays/lists, structs, nested structs or map fields. For example: - - Supported: - - `SELECT [1, 2, 3];` - - `SELECT ['1992-09-20 11:30:00.123456789', 'epoch'::TIMESTAMP]` +- The DuckDB accelerator does not support nested lists, or structs with nested structs/lists [field types](https://duckdb.org/docs/sql/data_types/overview). For example: - Unsupported: - `SELECT [['duck', 'goose', 'heron'], ['frog', 'toad']]` - - `SELECT {'x': 1, 'y': 2, 'z': 3}` +- The DuckDB accelerator does not support enum, dictionary, or map [field types](https://duckdb.org/docs/sql/data_types/overview). For example: + - Unsupported: - `SELECT MAP(['key1', 'key2', 'key3'], [10, 20, 30])` - - `SELECT ['duck', 'goose', 'heron'];` - - `SELECT [true, false];` - The DuckDB accelerator does not support `Decimal256` (76 digits), as it exceeds DuckDB's maximum Decimal width of 38 digits. -- Updating a dataset with DuckDB acceleration while the Spice Runtime is running (hot-reload) will cause DuckDB accelerator query federation to disable until the Runtime is restarted. +- Updating a dataset with DuckDB acceleration while the Spice Runtime is running (hot-reload) will cause the DuckDB accelerator query federation to disable until the Runtime is restarted. ::: diff --git a/spiceaidocs/docs/components/data-connectors/duckdb.md b/spiceaidocs/docs/components/data-connectors/duckdb.md index cfb6166e..4949af25 100644 --- a/spiceaidocs/docs/components/data-connectors/duckdb.md +++ b/spiceaidocs/docs/components/data-connectors/duckdb.md @@ -83,3 +83,16 @@ SELECT * FROM 'todos.json'; -- As a DuckDB function SELECT * FROM read_json('todos.json'); ``` + +:::warning[Limitations] + +- The DuckDB connector does not support nested lists, or structs with nested structs/lists [field types](https://duckdb.org/docs/sql/data_types/overview). For example: + - Unsupported: + - `SELECT [['duck', 'goose', 'heron'], ['frog', 'toad']]` +- The DuckDB connector does not support enum, dictionary, or map [field types](https://duckdb.org/docs/sql/data_types/overview). For example: + - Unsupported: + - `SELECT MAP(['key1', 'key2', 'key3'], [10, 20, 30])` +- The DuckDB connector does not support `Decimal256` (76 digits), as it exceeds DuckDB's maximum Decimal width of 38 digits. +- Updating a dataset with a DuckDB connector while the Spice Runtime is running (hot-reload) will cause the DuckDB connector query federation to disable until the Runtime is restarted. + +::: From fddc7873b4ab95b97aa767e9b4aab26ddd65c23a Mon Sep 17 00:00:00 2001 From: peasee <98815791+peasee@users.noreply.github.com> Date: Wed, 13 Nov 2024 13:15:42 +1000 Subject: [PATCH 2/3] docs: Clarify supported structs format --- spiceaidocs/docs/components/data-accelerators/duckdb.md | 3 +++ spiceaidocs/docs/components/data-connectors/duckdb.md | 3 +++ 2 files changed, 6 insertions(+) diff --git a/spiceaidocs/docs/components/data-accelerators/duckdb.md b/spiceaidocs/docs/components/data-accelerators/duckdb.md index e99dbf13..db11e1b8 100644 --- a/spiceaidocs/docs/components/data-accelerators/duckdb.md +++ b/spiceaidocs/docs/components/data-accelerators/duckdb.md @@ -37,8 +37,11 @@ datasets: :::warning[Limitations] - The DuckDB accelerator does not support nested lists, or structs with nested structs/lists [field types](https://duckdb.org/docs/sql/data_types/overview). For example: + - Supported: + - `SELECT {'x': 1, 'y': 2, 'z': 3}` - Unsupported: - `SELECT [['duck', 'goose', 'heron'], ['frog', 'toad']]` + - `SELECT {'x': [1, 2, 3]}` - The DuckDB accelerator does not support enum, dictionary, or map [field types](https://duckdb.org/docs/sql/data_types/overview). For example: - Unsupported: - `SELECT MAP(['key1', 'key2', 'key3'], [10, 20, 30])` diff --git a/spiceaidocs/docs/components/data-connectors/duckdb.md b/spiceaidocs/docs/components/data-connectors/duckdb.md index 4949af25..92b685e3 100644 --- a/spiceaidocs/docs/components/data-connectors/duckdb.md +++ b/spiceaidocs/docs/components/data-connectors/duckdb.md @@ -87,8 +87,11 @@ SELECT * FROM read_json('todos.json'); :::warning[Limitations] - The DuckDB connector does not support nested lists, or structs with nested structs/lists [field types](https://duckdb.org/docs/sql/data_types/overview). For example: + - Supported: + - `SELECT {'x': 1, 'y': 2, 'z': 3}` - Unsupported: - `SELECT [['duck', 'goose', 'heron'], ['frog', 'toad']]` + - `SELECT {'x': [1, 2, 3]}` - The DuckDB connector does not support enum, dictionary, or map [field types](https://duckdb.org/docs/sql/data_types/overview). For example: - Unsupported: - `SELECT MAP(['key1', 'key2', 'key3'], [10, 20, 30])` From 9d5c31a735d38020f93801a60200f43b158a8321 Mon Sep 17 00:00:00 2001 From: peasee <98815791+peasee@users.noreply.github.com> Date: Wed, 13 Nov 2024 13:28:49 +1000 Subject: [PATCH 3/3] docs: Remove federation limitation that does not apply to DuckDB connectors --- spiceaidocs/docs/components/data-connectors/duckdb.md | 1 - 1 file changed, 1 deletion(-) diff --git a/spiceaidocs/docs/components/data-connectors/duckdb.md b/spiceaidocs/docs/components/data-connectors/duckdb.md index 92b685e3..3875d8e9 100644 --- a/spiceaidocs/docs/components/data-connectors/duckdb.md +++ b/spiceaidocs/docs/components/data-connectors/duckdb.md @@ -96,6 +96,5 @@ SELECT * FROM read_json('todos.json'); - Unsupported: - `SELECT MAP(['key1', 'key2', 'key3'], [10, 20, 30])` - The DuckDB connector does not support `Decimal256` (76 digits), as it exceeds DuckDB's maximum Decimal width of 38 digits. -- Updating a dataset with a DuckDB connector while the Spice Runtime is running (hot-reload) will cause the DuckDB connector query federation to disable until the Runtime is restarted. :::