diff --git a/spiceaidocs/config.toml b/spiceaidocs/config.toml index c7333e8d..ef2aab6a 100644 --- a/spiceaidocs/config.toml +++ b/spiceaidocs/config.toml @@ -125,7 +125,7 @@ footer_about_disable = false # End user relevant links. These will show up on left side of footer and in the community page if you have one. [[params.links.developer]] name ="Twitter" - url = "https://twitter.com/SpiceAIHQ" + url = "https://twitter.com/spice_ai" icon = "fab fa-twitter" desc = "Follow us on Twitter to get the latest news!" # Developer relevant links. These will show up on right side of footer and in the community page if you have one. diff --git a/spiceaidocs/content/en/Connectors/_index.md b/spiceaidocs/content/en/Connectors/_index.md deleted file mode 100644 index 7bb2835d..00000000 --- a/spiceaidocs/content/en/Connectors/_index.md +++ /dev/null @@ -1,9 +0,0 @@ ---- -type: docs -title: "Data Connectors" -linkTitle: "Data Connectors" -description: '' -weight: 35 ---- - - diff --git a/spiceaidocs/content/en/_index.md b/spiceaidocs/content/en/_index.md index e58a75a0..5712cfe2 100644 --- a/spiceaidocs/content/en/_index.md +++ b/spiceaidocs/content/en/_index.md @@ -4,9 +4,10 @@ no_list: true --- # Spice + ## What is Spice? -**Spice** is a small, portable runtime that provides developers with a unified SQL query interface to locally accelerate and query data tables sourced from any database, data warehouse, or data lake. +**Spice** is a small, portable runtime that provides developers with a unified SQL query interface to locally materialize, accelerate, and query data tables sourced from any database, data warehouse, or data lake. Spice makes it easy to build data-driven and data-intensive applications by streamlining the use of data and machine learning (ML) in software. diff --git a/spiceaidocs/content/en/acknowledgements/_index.md b/spiceaidocs/content/en/acknowledgements/_index.md index 64c7d153..674ae2aa 100644 --- a/spiceaidocs/content/en/acknowledgements/_index.md +++ b/spiceaidocs/content/en/acknowledgements/_index.md @@ -2,8 +2,8 @@ type: docs title: Open Source Acknowledgements linkTitle: Acknowledgements -weight: 70 -description: "Spice.ai acknowledges the following open source projects for making this project possible:" +weight: 1000 +description: 'Spice.ai acknowledges the following open source projects for making this project possible:' no_list: true --- @@ -75,85 +75,85 @@ gopkg.in/yaml.v3, https://github.com/go-yaml/yaml/blob/v3.0.1/LICENSE, MIT ### Rust Crates -| name | version | authors | repository | license | license_file | description | -|------|---------|---------|------------|---------|--------------|-------------| -| name | version | authors | repository | license | license_file | description | -| arrow | 50.0.0 | Apache Arrow | https://github.com/apache/arrow-rs | Apache-2.0 | | Rust implementation of Apache Arrow | -| arrow-flight | 50.0.0 | Apache Arrow | https://github.com/apache/arrow-rs | Apache-2.0 | | Apache Arrow Flight | -| arrow-ipc | 49.0.0 | Apache Arrow | https://github.com/apache/arrow-rs | Apache-2.0 | | Support for the Arrow IPC format | -| arrow-ipc | 50.0.0 | Apache Arrow | https://github.com/apache/arrow-rs | Apache-2.0 | | Support for the Arrow IPC format | -| arrow-json | 50.0.0 | Apache Arrow | https://github.com/apache/arrow-rs | Apache-2.0 | | Support for parsing JSON format to and from the Arrow format | -| async-stream | 0.3.5 | Carl Lerche | https://github.com/tokio-rs/async-stream | MIT | | Asynchronous streams using async & await notation | -| async-trait | 0.1.77 | David Tolnay | https://github.com/dtolnay/async-trait | Apache-2.0 OR MIT | | Type erasure for async trait methods | -| aws-config | 1.1.8 | AWS Rust SDK Team |Russell Cohen | https://github.com/smithy-lang/smithy-rs | Apache-2.0 | | AWS SDK config and credential provider implementations. | -| aws-types | 1.1.8 | AWS Rust SDK Team |Russell Cohen | https://github.com/smithy-lang/smithy-rs | Apache-2.0 | | Cross-service types for the AWS SDK. | -| axum | 0.6.20 | | https://github.com/tokio-rs/axum | MIT | | Web framework that focuses on ergonomics and modularity | -| axum | 0.7.4 | | https://github.com/tokio-rs/axum | MIT | | Web framework that focuses on ergonomics and modularity | -| base64 | 0.13.1 | Alice Maz |Marshall Pierce | https://github.com/marshallpierce/rust-base64 | Apache-2.0 OR MIT | | encodes and decodes base64 as bytes or utf8 | -| base64 | 0.21.7 | Alice Maz |Marshall Pierce | https://github.com/marshallpierce/rust-base64 | Apache-2.0 OR MIT | | encodes and decodes base64 as bytes or utf8 | -| base64 | 0.22.0 | Alice Maz |Marshall Pierce | https://github.com/marshallpierce/rust-base64 | Apache-2.0 OR MIT | | encodes and decodes base64 as bytes or utf8 | -| bb8 | 0.7.1 | Dirkjan Ochtman |Kyle Huey | https://github.com/djc/bb8 | MIT | | Full-featured async (tokio-based) connection pool (like r2d2) | -| bb8-postgres | 0.7.0 | Dirkjan Ochtman |Kyle Huey | https://github.com/khuey/bb8 | MIT | | Full-featured async (tokio-based) postgres connection pool (like r2d2) | -| bigdecimal | 0.3.1 | Andrew Kubera | https://github.com/akubera/bigdecimal-rs | Apache-2.0 OR MIT | | Arbitrary precision decimal numbers | -| bigdecimal | 0.4.3 | Andrew Kubera | https://github.com/akubera/bigdecimal-rs | Apache-2.0 OR MIT | | Arbitrary precision decimal numbers | -| bytes | 1.5.0 | Carl Lerche |Sean McArthur | https://github.com/tokio-rs/bytes | MIT | | Types and traits for working with bytes | -| clap | 4.5.2 | | https://github.com/clap-rs/clap | Apache-2.0 OR MIT | | A simple to use, efficient, and full-featured Command Line Argument Parser | -| csv | 1.3.0 | Andrew Gallant | https://github.com/BurntSushi/rust-csv | MIT OR Unlicense | | Fast CSV parsing with support for serde. | -| datafusion | 35.0.0 | Apache Arrow | https://github.com/apache/arrow-datafusion | Apache-2.0 | | DataFusion is an in-memory query engine that uses Apache Arrow as the memory model | -| deltalake | 0.17.0 | Qingping Hou | https://github.com/delta-io/delta.rs | Apache-2.0 | | Native Delta Lake implementation in Rust | -| dirs | 5.0.1 | Simon Ochsenreither | https://github.com/soc/dirs-rs | Apache-2.0 OR MIT | | A tiny low-level library that provides platform-specific standard locations of directories for config, cache and other data on Linux, Windows, macOS and Redox by leveraging the mechanisms defined by the XDG base/user directory specifications on Linux, the Known Folder API on Windows, and the Standard Directory guidelines on macOS. | -| duckdb | 0.10.0 | wangfenjin | https://github.com/wangfenjin/duckdb-rs | MIT | | Ergonomic wrapper for DuckDB | -| fundu | 2.0.0 | Joining7943 | https://github.com/fundu-rs/fundu | MIT | | Configurable, precise and fast rust string parser to a Duration | -| futures | 0.3.30 | | https://github.com/rust-lang/futures-rs | Apache-2.0 OR MIT | | An implementation of futures and streams featuring zero allocations, composability, and iterator-like interfaces. | -| futures-core | 0.3.30 | | https://github.com/rust-lang/futures-rs | Apache-2.0 OR MIT | | The core traits and types in for the `futures` library. | -| indexmap | 1.9.3 | | https://github.com/bluss/indexmap | Apache-2.0 OR MIT | | A hash table with consistent order and fast iteration. | -| indexmap | 2.2.5 | | https://github.com/indexmap-rs/indexmap | Apache-2.0 OR MIT | | A hash table with consistent order and fast iteration. | -| itertools | 0.10.5 | bluss | https://github.com/rust-itertools/itertools | Apache-2.0 OR MIT | | Extra iterator adaptors, iterator methods, free functions, and macros. | -| itertools | 0.11.0 | bluss | https://github.com/rust-itertools/itertools | Apache-2.0 OR MIT | | Extra iterator adaptors, iterator methods, free functions, and macros. | -| itertools | 0.12.1 | bluss | https://github.com/rust-itertools/itertools | Apache-2.0 OR MIT | | Extra iterator adaptors, iterator methods, free functions, and macros. | -| keyring | 2.3.2 | Walther Chen |Daniel Brotsky | https://github.com/hwchen/keyring-rs.git | Apache-2.0 OR MIT | | Cross-platform library for managing passwords/credentials | -| metrics | 0.22.1 | Toby Lawrence | https://github.com/metrics-rs/metrics | MIT | | A lightweight metrics facade. | -| metrics-exporter-prometheus | 0.13.1 | Toby Lawrence | https://github.com/metrics-rs/metrics | MIT | | A metrics-compatible exporter for sending metrics to Prometheus. | -| ndarray | 0.15.6 | Ulrik Sverdrup bluss|Jim Turner | https://github.com/rust-ndarray/ndarray | Apache-2.0 OR MIT | | An n-dimensional array for general elements and for numerics. Lightweight array views and slicing; views support chunking and splitting. | -| ndarray-npy | 0.8.1 | Jim Turner | https://github.com/jturner314/ndarray-npy | Apache-2.0 OR MIT | | .npy and .npz file format support for ndarray | -| notify | 6.1.1 | Félix Saparelli |Daniel Faust |Aron Heinecke | https://github.com/notify-rs/notify.git | CC0-1.0 | | Cross-platform filesystem notification library | -| object_store | 0.9.1 | | https://github.com/apache/arrow-rs/tree/master/object_store | Apache-2.0 OR MIT | | A generic object store interface for uniformly interacting with AWS S3, Google Cloud Storage, Azure Blob Storage and local files. | -| once_cell | 1.19.0 | Aleksey Kladov | https://github.com/matklad/once_cell | Apache-2.0 OR MIT | | Single assignment cells and lazy values. | -| opentelemetry-proto | 0.4.0 | | https://github.com/open-telemetry/opentelemetry-rust/tree/main/opentelemetry-proto | Apache-2.0 | | Protobuf generated files and transformations. | -| parquet | 49.0.0 | Apache Arrow | https://github.com/apache/arrow-rs | Apache-2.0 | | Apache Parquet implementation in Rust | -| parquet | 50.0.0 | Apache Arrow | https://github.com/apache/arrow-rs | Apache-2.0 | | Apache Parquet implementation in Rust | -| prost | 0.11.9 | Dan Burkert |Lucio Franco | https://github.com/tokio-rs/prost | Apache-2.0 | | A Protocol Buffers implementation for the Rust Language. | -| prost | 0.12.3 | Dan Burkert |Lucio Franco | https://github.com/tokio-rs/prost | Apache-2.0 | | A Protocol Buffers implementation for the Rust Language. | -| quick-xml | 0.23.1 | | https://github.com/tafia/quick-xml | MIT | | High performance xml reader and writer | -| quick-xml | 0.31.0 | | https://github.com/tafia/quick-xml | MIT | | High performance xml reader and writer | -| r2d2 | 0.8.10 | Steven Fackler | https://github.com/sfackler/r2d2 | Apache-2.0 OR MIT | | A generic connection pool | -| regex | 1.10.3 | The Rust Project Developers|Andrew Gallant | https://github.com/rust-lang/regex | Apache-2.0 OR MIT | | An implementation of regular expressions for Rust. This implementation uses finite automata and guarantees linear time matching on all inputs. | -| reqwest | 0.11.26 | Sean McArthur | https://github.com/seanmonstar/reqwest | Apache-2.0 OR MIT | | higher level HTTP client library | -| rustls | 0.19.1 | Joseph Birr-Pixton | https://github.com/ctz/rustls | Apache-2.0 OR ISC OR MIT | | Rustls is a modern TLS library written in Rust. | -| rustls | 0.21.10 | | https://github.com/rustls/rustls | Apache-2.0 OR ISC OR MIT | | Rustls is a modern TLS library written in Rust. | -| rustls-native-certs | 0.5.0 | Joseph Birr-Pixton | https://github.com/ctz/rustls-native-certs | Apache-2.0 OR ISC OR MIT | | rustls-native-certs allows rustls to use the platform native certificate store | -| rustls-native-certs | 0.6.3 | | https://github.com/ctz/rustls-native-certs | Apache-2.0 OR ISC OR MIT | | rustls-native-certs allows rustls to use the platform native certificate store | -| rustls-pemfile | 1.0.4 | | https://github.com/rustls/pemfile | Apache-2.0 OR ISC OR MIT | | Basic .pem file parser for keys and certificates | -| rustyline | 13.0.0 | Katsu Kawakami | https://github.com/kkawakam/rustyline | MIT | | Rustyline, a readline implementation based on Antirez's Linenoise | -| sea-query | 0.30.7 | Chris Tsang |Billy Chan |Ivan Krivosheev | https://github.com/SeaQL/sea-query | Apache-2.0 OR MIT | | 🔱 A dynamic query builder for MySQL, Postgres and SQLite | -| secrecy | 0.8.0 | Tony Arcieri | https://github.com/iqlusioninc/crates/tree/main/secrecy | Apache-2.0 OR MIT | | Wrapper types and traits for secret management which help ensure they aren't accidentally copied, logged, or otherwise exposed (as much as possible), and also ensure secrets are securely wiped from memory when dropped. | -| serde | 1.0.197 | Erick Tryzelaar |David Tolnay | https://github.com/serde-rs/serde | Apache-2.0 OR MIT | | A generic serialization/deserialization framework | -| serde_json | 1.0.114 | Erick Tryzelaar |David Tolnay | https://github.com/serde-rs/json | Apache-2.0 OR MIT | | A JSON serialization file format | -| serde_yaml | 0.9.32 | David Tolnay | https://github.com/dtolnay/serde-yaml | Apache-2.0 OR MIT | | YAML data format for Serde | -| snafu | 0.7.5 | Jake Goulding | https://github.com/shepmaster/snafu | Apache-2.0 OR MIT | | An ergonomic error handling library | -| snafu | 0.8.2 | Jake Goulding | https://github.com/shepmaster/snafu | Apache-2.0 OR MIT | | An ergonomic error handling library | -| time | 0.3.34 | Jacob Pratt |Time contributors | https://github.com/time-rs/time | Apache-2.0 OR MIT | | Date and time library. Fully interoperable with the standard library. Mostly compatible with #![no_std]. | -| tokio | 1.36.0 | Tokio Contributors | https://github.com/tokio-rs/tokio | MIT | | An event-driven, non-blocking I/O platform for writing asynchronous I/O backed applications. | -| tokio-postgres | 0.7.10 | Steven Fackler | https://github.com/sfackler/rust-postgres | Apache-2.0 OR MIT | | A native, asynchronous PostgreSQL client | -| tokio-stream | 0.1.14 | Tokio Contributors | https://github.com/tokio-rs/tokio | MIT | | Utilities to work with `Stream` and `tokio`. | -| toml | 0.8.11 | Alex Crichton | https://github.com/toml-rs/toml | Apache-2.0 OR MIT | | A native Rust encoder and decoder of TOML-formatted files and streams. Provides implementations of the standard Serialize/Deserialize traits for TOML data to facilitate deserializing and serializing Rust structures. | -| tonic | 0.9.2 | Lucio Franco | https://github.com/hyperium/tonic | MIT | | A gRPC over HTTP/2 implementation focused on high performance, interoperability, and flexibility. | -| tonic | 0.10.2 | Lucio Franco | https://github.com/hyperium/tonic | MIT | | A gRPC over HTTP/2 implementation focused on high performance, interoperability, and flexibility. | -| tonic | 0.11.0 | Lucio Franco | https://github.com/hyperium/tonic | MIT | | A gRPC over HTTP/2 implementation focused on high performance, interoperability, and flexibility. | -| tonic-health | 0.11.0 | James Nugent | https://github.com/hyperium/tonic | MIT | | Health Checking module of `tonic` gRPC implementation. | -| tracing | 0.1.40 | Eliza Weisman |Tokio Contributors | https://github.com/tokio-rs/tracing | MIT | | Application-level tracing for Rust. | -| tracing-subscriber | 0.3.18 | Eliza Weisman |David Barsky |Tokio Contributors | https://github.com/tokio-rs/tracing | MIT | | Utilities for implementing and composing `tracing` subscribers. | -| tract-core | 0.21.1 | Mathieu Poumeyrol | https://github.com/snipsco/tract | Apache-2.0 OR MIT | | Tiny, no-nonsense, self contained, TensorFlow and ONNX inference | -| tract-onnx | 0.21.1 | Mathieu Poumeyrol | https://github.com/snipsco/tract | Apache-2.0 OR MIT | | Tiny, no-nonsense, self contained, TensorFlow and ONNX inference | -| url | 2.5.0 | The rust-url developers | https://github.com/servo/rust-url | Apache-2.0 OR MIT | | URL library for Rust, based on the WHATWG URL Standard | -| uuid | 1.7.0 | Ashley Mannix|Christopher Armstrong|Dylan DPC|Hunar Roop Kahlon | https://github.com/uuid-rs/uuid | Apache-2.0 OR MIT | | A library to generate and parse UUIDs. | +| name | version | authors | repository | license | license_file | description | +| --------------------------- | ------- | -------------------------------------------- | ---------------------------------------------------------------------------------- | --------------------------------------------- | --------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------- | -------------------------------------- | +| name | version | authors | repository | license | license_file | description | +| arrow | 50.0.0 | Apache Arrow | https://github.com/apache/arrow-rs | Apache-2.0 | | Rust implementation of Apache Arrow | +| arrow-flight | 50.0.0 | Apache Arrow | https://github.com/apache/arrow-rs | Apache-2.0 | | Apache Arrow Flight | +| arrow-ipc | 49.0.0 | Apache Arrow | https://github.com/apache/arrow-rs | Apache-2.0 | | Support for the Arrow IPC format | +| arrow-ipc | 50.0.0 | Apache Arrow | https://github.com/apache/arrow-rs | Apache-2.0 | | Support for the Arrow IPC format | +| arrow-json | 50.0.0 | Apache Arrow | https://github.com/apache/arrow-rs | Apache-2.0 | | Support for parsing JSON format to and from the Arrow format | +| async-stream | 0.3.5 | Carl Lerche | https://github.com/tokio-rs/async-stream | MIT | | Asynchronous streams using async & await notation | +| async-trait | 0.1.77 | David Tolnay | https://github.com/dtolnay/async-trait | Apache-2.0 OR MIT | | Type erasure for async trait methods | +| aws-config | 1.1.8 | AWS Rust SDK Team | Russell Cohen | https://github.com/smithy-lang/smithy-rs | Apache-2.0 | | AWS SDK config and credential provider implementations. | +| aws-types | 1.1.8 | AWS Rust SDK Team | Russell Cohen | https://github.com/smithy-lang/smithy-rs | Apache-2.0 | | Cross-service types for the AWS SDK. | +| axum | 0.6.20 | | https://github.com/tokio-rs/axum | MIT | | Web framework that focuses on ergonomics and modularity | +| axum | 0.7.4 | | https://github.com/tokio-rs/axum | MIT | | Web framework that focuses on ergonomics and modularity | +| base64 | 0.13.1 | Alice Maz | Marshall Pierce | https://github.com/marshallpierce/rust-base64 | Apache-2.0 OR MIT | | encodes and decodes base64 as bytes or utf8 | +| base64 | 0.21.7 | Alice Maz | Marshall Pierce | https://github.com/marshallpierce/rust-base64 | Apache-2.0 OR MIT | | encodes and decodes base64 as bytes or utf8 | +| base64 | 0.22.0 | Alice Maz | Marshall Pierce | https://github.com/marshallpierce/rust-base64 | Apache-2.0 OR MIT | | encodes and decodes base64 as bytes or utf8 | +| bb8 | 0.7.1 | Dirkjan Ochtman | Kyle Huey | https://github.com/djc/bb8 | MIT | | Full-featured async (tokio-based) connection pool (like r2d2) | +| bb8-postgres | 0.7.0 | Dirkjan Ochtman | Kyle Huey | https://github.com/khuey/bb8 | MIT | | Full-featured async (tokio-based) postgres connection pool (like r2d2) | +| bigdecimal | 0.3.1 | Andrew Kubera | https://github.com/akubera/bigdecimal-rs | Apache-2.0 OR MIT | | Arbitrary precision decimal numbers | +| bigdecimal | 0.4.3 | Andrew Kubera | https://github.com/akubera/bigdecimal-rs | Apache-2.0 OR MIT | | Arbitrary precision decimal numbers | +| bytes | 1.5.0 | Carl Lerche | Sean McArthur | https://github.com/tokio-rs/bytes | MIT | | Types and traits for working with bytes | +| clap | 4.5.2 | | https://github.com/clap-rs/clap | Apache-2.0 OR MIT | | A simple to use, efficient, and full-featured Command Line Argument Parser | +| csv | 1.3.0 | Andrew Gallant | https://github.com/BurntSushi/rust-csv | MIT OR Unlicense | | Fast CSV parsing with support for serde. | +| datafusion | 35.0.0 | Apache Arrow | https://github.com/apache/arrow-datafusion | Apache-2.0 | | DataFusion is an in-memory query engine that uses Apache Arrow as the memory model | +| deltalake | 0.17.0 | Qingping Hou | https://github.com/delta-io/delta.rs | Apache-2.0 | | Native Delta Lake implementation in Rust | +| dirs | 5.0.1 | Simon Ochsenreither | https://github.com/soc/dirs-rs | Apache-2.0 OR MIT | | A tiny low-level library that provides platform-specific standard locations of directories for config, cache and other data on Linux, Windows, macOS and Redox by leveraging the mechanisms defined by the XDG base/user directory specifications on Linux, the Known Folder API on Windows, and the Standard Directory guidelines on macOS. | +| duckdb | 0.10.0 | wangfenjin | https://github.com/wangfenjin/duckdb-rs | MIT | | Ergonomic wrapper for DuckDB | +| fundu | 2.0.0 | Joining7943 | https://github.com/fundu-rs/fundu | MIT | | Configurable, precise and fast rust string parser to a Duration | +| futures | 0.3.30 | | https://github.com/rust-lang/futures-rs | Apache-2.0 OR MIT | | An implementation of futures and streams featuring zero allocations, composability, and iterator-like interfaces. | +| futures-core | 0.3.30 | | https://github.com/rust-lang/futures-rs | Apache-2.0 OR MIT | | The core traits and types in for the `futures` library. | +| indexmap | 1.9.3 | | https://github.com/bluss/indexmap | Apache-2.0 OR MIT | | A hash table with consistent order and fast iteration. | +| indexmap | 2.2.5 | | https://github.com/indexmap-rs/indexmap | Apache-2.0 OR MIT | | A hash table with consistent order and fast iteration. | +| itertools | 0.10.5 | bluss | https://github.com/rust-itertools/itertools | Apache-2.0 OR MIT | | Extra iterator adaptors, iterator methods, free functions, and macros. | +| itertools | 0.11.0 | bluss | https://github.com/rust-itertools/itertools | Apache-2.0 OR MIT | | Extra iterator adaptors, iterator methods, free functions, and macros. | +| itertools | 0.12.1 | bluss | https://github.com/rust-itertools/itertools | Apache-2.0 OR MIT | | Extra iterator adaptors, iterator methods, free functions, and macros. | +| keyring | 2.3.2 | Walther Chen | Daniel Brotsky | https://github.com/hwchen/keyring-rs.git | Apache-2.0 OR MIT | | Cross-platform library for managing passwords/credentials | +| metrics | 0.22.1 | Toby Lawrence | https://github.com/metrics-rs/metrics | MIT | | A lightweight metrics facade. | +| metrics-exporter-prometheus | 0.13.1 | Toby Lawrence | https://github.com/metrics-rs/metrics | MIT | | A metrics-compatible exporter for sending metrics to Prometheus. | +| ndarray | 0.15.6 | Ulrik Sverdrup bluss | Jim Turner | https://github.com/rust-ndarray/ndarray | Apache-2.0 OR MIT | | An n-dimensional array for general elements and for numerics. Lightweight array views and slicing; views support chunking and splitting. | +| ndarray-npy | 0.8.1 | Jim Turner | https://github.com/jturner314/ndarray-npy | Apache-2.0 OR MIT | | .npy and .npz file format support for ndarray | +| notify | 6.1.1 | Félix Saparelli | Daniel Faust | Aron Heinecke | https://github.com/notify-rs/notify.git | CC0-1.0 | | Cross-platform filesystem notification library | +| object_store | 0.9.1 | | https://github.com/apache/arrow-rs/tree/master/object_store | Apache-2.0 OR MIT | | A generic object store interface for uniformly interacting with AWS S3, Google Cloud Storage, Azure Blob Storage and local files. | +| once_cell | 1.19.0 | Aleksey Kladov | https://github.com/matklad/once_cell | Apache-2.0 OR MIT | | Single assignment cells and lazy values. | +| opentelemetry-proto | 0.4.0 | | https://github.com/open-telemetry/opentelemetry-rust/tree/main/opentelemetry-proto | Apache-2.0 | | Protobuf generated files and transformations. | +| parquet | 49.0.0 | Apache Arrow | https://github.com/apache/arrow-rs | Apache-2.0 | | Apache Parquet implementation in Rust | +| parquet | 50.0.0 | Apache Arrow | https://github.com/apache/arrow-rs | Apache-2.0 | | Apache Parquet implementation in Rust | +| prost | 0.11.9 | Dan Burkert | Lucio Franco | https://github.com/tokio-rs/prost | Apache-2.0 | | A Protocol Buffers implementation for the Rust Language. | +| prost | 0.12.3 | Dan Burkert | Lucio Franco | https://github.com/tokio-rs/prost | Apache-2.0 | | A Protocol Buffers implementation for the Rust Language. | +| quick-xml | 0.23.1 | | https://github.com/tafia/quick-xml | MIT | | High performance xml reader and writer | +| quick-xml | 0.31.0 | | https://github.com/tafia/quick-xml | MIT | | High performance xml reader and writer | +| r2d2 | 0.8.10 | Steven Fackler | https://github.com/sfackler/r2d2 | Apache-2.0 OR MIT | | A generic connection pool | +| regex | 1.10.3 | The Rust Project Developers | Andrew Gallant | https://github.com/rust-lang/regex | Apache-2.0 OR MIT | | An implementation of regular expressions for Rust. This implementation uses finite automata and guarantees linear time matching on all inputs. | +| reqwest | 0.11.26 | Sean McArthur | https://github.com/seanmonstar/reqwest | Apache-2.0 OR MIT | | higher level HTTP client library | +| rustls | 0.19.1 | Joseph Birr-Pixton | https://github.com/ctz/rustls | Apache-2.0 OR ISC OR MIT | | Rustls is a modern TLS library written in Rust. | +| rustls | 0.21.10 | | https://github.com/rustls/rustls | Apache-2.0 OR ISC OR MIT | | Rustls is a modern TLS library written in Rust. | +| rustls-native-certs | 0.5.0 | Joseph Birr-Pixton | https://github.com/ctz/rustls-native-certs | Apache-2.0 OR ISC OR MIT | | rustls-native-certs allows rustls to use the platform native certificate store | +| rustls-native-certs | 0.6.3 | | https://github.com/ctz/rustls-native-certs | Apache-2.0 OR ISC OR MIT | | rustls-native-certs allows rustls to use the platform native certificate store | +| rustls-pemfile | 1.0.4 | | https://github.com/rustls/pemfile | Apache-2.0 OR ISC OR MIT | | Basic .pem file parser for keys and certificates | +| rustyline | 13.0.0 | Katsu Kawakami | https://github.com/kkawakam/rustyline | MIT | | Rustyline, a readline implementation based on Antirez's Linenoise | +| sea-query | 0.30.7 | Chris Tsang | Billy Chan | Ivan Krivosheev | https://github.com/SeaQL/sea-query | Apache-2.0 OR MIT | | 🔱 A dynamic query builder for MySQL, Postgres and SQLite | +| secrecy | 0.8.0 | Tony Arcieri | https://github.com/iqlusioninc/crates/tree/main/secrecy | Apache-2.0 OR MIT | | Wrapper types and traits for secret management which help ensure they aren't accidentally copied, logged, or otherwise exposed (as much as possible), and also ensure secrets are securely wiped from memory when dropped. | +| serde | 1.0.197 | Erick Tryzelaar | David Tolnay | https://github.com/serde-rs/serde | Apache-2.0 OR MIT | | A generic serialization/deserialization framework | +| serde_json | 1.0.114 | Erick Tryzelaar | David Tolnay | https://github.com/serde-rs/json | Apache-2.0 OR MIT | | A JSON serialization file format | +| serde_yaml | 0.9.32 | David Tolnay | https://github.com/dtolnay/serde-yaml | Apache-2.0 OR MIT | | YAML data format for Serde | +| snafu | 0.7.5 | Jake Goulding | https://github.com/shepmaster/snafu | Apache-2.0 OR MIT | | An ergonomic error handling library | +| snafu | 0.8.2 | Jake Goulding | https://github.com/shepmaster/snafu | Apache-2.0 OR MIT | | An ergonomic error handling library | +| time | 0.3.34 | Jacob Pratt | Time contributors | https://github.com/time-rs/time | Apache-2.0 OR MIT | | Date and time library. Fully interoperable with the standard library. Mostly compatible with #![no_std]. | +| tokio | 1.36.0 | Tokio Contributors | https://github.com/tokio-rs/tokio | MIT | | An event-driven, non-blocking I/O platform for writing asynchronous I/O backed applications. | +| tokio-postgres | 0.7.10 | Steven Fackler | https://github.com/sfackler/rust-postgres | Apache-2.0 OR MIT | | A native, asynchronous PostgreSQL client | +| tokio-stream | 0.1.14 | Tokio Contributors | https://github.com/tokio-rs/tokio | MIT | | Utilities to work with `Stream` and `tokio`. | +| toml | 0.8.11 | Alex Crichton | https://github.com/toml-rs/toml | Apache-2.0 OR MIT | | A native Rust encoder and decoder of TOML-formatted files and streams. Provides implementations of the standard Serialize/Deserialize traits for TOML data to facilitate deserializing and serializing Rust structures. | +| tonic | 0.9.2 | Lucio Franco | https://github.com/hyperium/tonic | MIT | | A gRPC over HTTP/2 implementation focused on high performance, interoperability, and flexibility. | +| tonic | 0.10.2 | Lucio Franco | https://github.com/hyperium/tonic | MIT | | A gRPC over HTTP/2 implementation focused on high performance, interoperability, and flexibility. | +| tonic | 0.11.0 | Lucio Franco | https://github.com/hyperium/tonic | MIT | | A gRPC over HTTP/2 implementation focused on high performance, interoperability, and flexibility. | +| tonic-health | 0.11.0 | James Nugent | https://github.com/hyperium/tonic | MIT | | Health Checking module of `tonic` gRPC implementation. | +| tracing | 0.1.40 | Eliza Weisman | Tokio Contributors | https://github.com/tokio-rs/tracing | MIT | | Application-level tracing for Rust. | +| tracing-subscriber | 0.3.18 | Eliza Weisman | David Barsky | Tokio Contributors | https://github.com/tokio-rs/tracing | MIT | | Utilities for implementing and composing `tracing` subscribers. | +| tract-core | 0.21.1 | Mathieu Poumeyrol | https://github.com/snipsco/tract | Apache-2.0 OR MIT | | Tiny, no-nonsense, self contained, TensorFlow and ONNX inference | +| tract-onnx | 0.21.1 | Mathieu Poumeyrol | https://github.com/snipsco/tract | Apache-2.0 OR MIT | | Tiny, no-nonsense, self contained, TensorFlow and ONNX inference | +| url | 2.5.0 | The rust-url developers | https://github.com/servo/rust-url | Apache-2.0 OR MIT | | URL library for Rust, based on the WHATWG URL Standard | +| uuid | 1.7.0 | Ashley Mannix | Christopher Armstrong | Dylan DPC | Hunar Roop Kahlon | https://github.com/uuid-rs/uuid | Apache-2.0 OR MIT | | A library to generate and parse UUIDs. | diff --git a/spiceaidocs/content/en/cli/_index.md b/spiceaidocs/content/en/cli/_index.md index 2665bbe5..80e760b6 100644 --- a/spiceaidocs/content/en/cli/_index.md +++ b/spiceaidocs/content/en/cli/_index.md @@ -1,9 +1,9 @@ --- type: docs -title: "Spice.ai CLI documentation" -linkTitle: "CLI" -weight: 60 -description: "Detailed documentation on the Spice.ai CLI" +title: 'Spice.ai CLI documentation' +linkTitle: 'CLI' +weight: 100 +description: 'Detailed documentation on the Spice.ai CLI' --- The Spice.ai CLI is a set of commands to create and manage Spice.ai pods and interact with the Spice.ai runtime. @@ -45,14 +45,13 @@ spice add spiceai/quickstart Common commands are: -| Command | Description | -| ----------------- | ------------------------------------------------------------------- | -| spice add | Add Pod - adds a pod to the project | -| spice run | Run Spice - starts the Spice runtime, installing if necessary | -| spice version | Spice CLI version | -| spice help | Help about any command | -| spice upgrade | Upgrades the Spice CLI to the latest release | - +| Command | Description | +| ------------- | ------------------------------------------------------------- | +| spice add | Add Pod - adds a pod to the project | +| spice run | Run Spice - starts the Spice runtime, installing if necessary | +| spice version | Spice CLI version | +| spice help | Help about any command | +| spice upgrade | Upgrades the Spice CLI to the latest release | See [Spice CLI command reference]({{}}) for the full list of available commands. diff --git a/spiceaidocs/content/en/flight-sql/DBeaver/_index.md b/spiceaidocs/content/en/clients/DBeaver/_index.md similarity index 100% rename from spiceaidocs/content/en/flight-sql/DBeaver/_index.md rename to spiceaidocs/content/en/clients/DBeaver/_index.md diff --git a/spiceaidocs/content/en/clients/_index.md b/spiceaidocs/content/en/clients/_index.md new file mode 100644 index 00000000..ef707924 --- /dev/null +++ b/spiceaidocs/content/en/clients/_index.md @@ -0,0 +1,7 @@ +--- +type: docs +title: 'Clients and Tools' +linkTitle: 'Clients and Tools' +weight: 110 +description: 'Client and tools' +--- diff --git a/spiceaidocs/content/en/flight-sql/_index.md b/spiceaidocs/content/en/clients/arrow-flight-sql/_index.md similarity index 100% rename from spiceaidocs/content/en/flight-sql/_index.md rename to spiceaidocs/content/en/clients/arrow-flight-sql/_index.md diff --git a/spiceaidocs/content/en/data-accelerators/_index.md b/spiceaidocs/content/en/data-accelerators/_index.md new file mode 100644 index 00000000..c4799bec --- /dev/null +++ b/spiceaidocs/content/en/data-accelerators/_index.md @@ -0,0 +1,33 @@ +--- +type: docs +title: 'Data Accelerators' +linkTitle: 'Data Accelerators' +description: '' +weight: 80 +--- + +Data sourced by Data Connectors can be locally materialized and accelerated using a Data Accelerator. + +Acceleration is enabled on a dataset by setting the acceleration configuration. E.g. + +```yaml +datasets: + - name: accelerated_dataset + acceleration: + enabled: true +``` + +For the complete reference specification see [datasets]({{}}). + +By default, datasets will be locally materialized using in-memory Arrow records. + +Data Accelerators using DuckDB, SQLite, or PostgreSQL engines can be used to materialize data in files or attached databases. + +Currently supported Data Accelerators include: + +| Engine Name | Description | Status | Engine Modes | +| ---------------------------------------------------- | ----------------------- | ------ | ---------------- | +| `arrow` | In-Memory Arrow Records | Alpha | `memory` | +| `duckdb` | Embedded DuckDB | Alpha | `memory`, `file` | +| `sqlite` | Embedded SQLite | Alpha | `memory`, `file` | +| [`postgres`]({{}}) | Attached PostgreSQL | Alpha | | diff --git a/spiceaidocs/content/en/data-accelerators/postgres/_index.md b/spiceaidocs/content/en/data-accelerators/postgres/_index.md new file mode 100644 index 00000000..deb68af0 --- /dev/null +++ b/spiceaidocs/content/en/data-accelerators/postgres/_index.md @@ -0,0 +1,66 @@ +--- +type: docs +title: 'PostgreSQL Data Accelerator' +linkTitle: 'PostgreSQL Data Accelerator' +description: 'PostgreSQL Data Accelerator Documentation' +--- + +To use PostgreSQL as Data Accelerator, specify `postgres` as the `engine` for acceleration. + +```yaml +datasets: + - from: spiceai:path.to.my_dataset + name: my_dataset + acceleration: + engine: postgres +``` + +## Configuration + +The connection to PostgreSQL can be configured by providing the following `params`: + +- `pg_host`: The hostname of the PostgreSQL server. +- `pg_port`: The port of the PostgreSQL server. +- `pg_db`: The name of the database to connect to. +- `pg_user`: The username to connect with. +- `pg_pass_key`: The secret key containing the password to connect with. +- `pg_pass`: The plain-text password to connect with, ignored if `pg_pass_key` is provided. + +Configuration `params` are provided either in the top level `dataset` for a dataset source and federated SQL query, or in the `acceleration` section for a data store. + +```yaml +datasets: + - from: spiceai:path.to.my_dataset + name: my_dataset + acceleration: + engine: postgres + params: + pg_host: localhost + pg_port: 5432 + pg_db: my_database + pg_user: my_user + pg_pass_key: my_secret +``` + +Additionally, an `engine_secret` may be provided when configuring a PostgreSQL data store to allow for using a different secret store to specify the password for a dataset using PostgreSQL as both the data source and data store. + +```yaml +datasets: + - from: spiceai:path.to.my_dataset + name: my_dataset + params: + pg_host: localhost + pg_port: 5432 + pg_db: data_store + pg_user: my_user + pg_pass_key: my_secret + acceleration: + engine: postgres + engine_secret: pg_backend + params: + pg_host: localhost + pg_port: 5433 + pg_db: data_store + pg_user: my_user + pg_pass_key: my_secret +``` diff --git a/spiceaidocs/content/en/data-connectors/_index.md b/spiceaidocs/content/en/data-connectors/_index.md new file mode 100644 index 00000000..6784823d --- /dev/null +++ b/spiceaidocs/content/en/data-connectors/_index.md @@ -0,0 +1,22 @@ +--- +type: docs +title: 'Data Connectors' +linkTitle: 'Data Connectors' +description: '' +weight: 70 +--- + +Data Connectors provide connections to databases, data warehouses, and data lakes for federated SQL queries and data replication. + +Currently supported Data Connectors include: + +| Name | Description | Status | Protocol/Format | Refresh Modes | Supports Inserts | +| ------------ | ----------- | ------------ | ---------------- | ---------------- | ---------------- | +| `databricks` | Databricks | Alpha | Delta Lake | `full` | ❌ | +| `postgres` | PostgreSQL | Alpha | | `full` | ✅ | +| `spiceai` | Spice.ai | Alpha | Arrow Flight | `append`, `full` | ✅ | +| `s3` | S3 | Alpha | Parquet | `full` | ❌ | +| `dremio` | Dremio | Alpha | Arrow Flight SQL | `full` | ❌ | +| `snowflake` | Snowflake | Coming soon! | Arrow Flight SQL | `full` | ❌ | +| `bigquery` | BigQuery | Coming soon! | Arrow Flight SQL | `full` | ❌ | +| `mysql` | MySQL | Coming soon! | | `full` | ❌ | diff --git a/spiceaidocs/content/en/Connectors/Postgres/_index.md b/spiceaidocs/content/en/data-connectors/postgres/_index.md similarity index 55% rename from spiceaidocs/content/en/Connectors/Postgres/_index.md rename to spiceaidocs/content/en/data-connectors/postgres/_index.md index ecf82fb6..d72a7474 100644 --- a/spiceaidocs/content/en/Connectors/Postgres/_index.md +++ b/spiceaidocs/content/en/data-connectors/postgres/_index.md @@ -1,12 +1,10 @@ --- type: docs -title: "PostgreSQL" -linkTitle: "PostgreSQL" -description: 'PostgreSQL reference' +title: 'PostgreSQL Data Connector' +linkTitle: 'PostgreSQL Data Connector' +description: 'PostgreSQL Data Connector Documentation' --- -PostgreSQL can be used by the Spice runtime as a dataset source, a data store, or for federated SQL query. - ## Dataset Source/Federated SQL Query To use PostgreSQL as a dataset source or for federated SQL query, specify `postgres` as the selector in the `from` value for the dataset. @@ -17,18 +15,6 @@ datasets: name: my_dataset ``` -## Data Store - -To use PostgreSQL as a data store for dataset acceleration, specify `postgres` as the `engine` for the dataset. - -```yaml -datasets: - - from: spiceai:path.to.my_dataset - name: my_dataset - acceleration: - engine: postgres -``` - ## Configuration The connection to PostgreSQL can be configured by providing the following `params`: @@ -42,34 +28,16 @@ The connection to PostgreSQL can be configured by providing the following `param Configuration `params` are provided either in the top level `dataset` for a dataset source and federated SQL query, or in the `acceleration` section for a data store. -### Dataset Source/Federated SQL Query - ```yaml datasets: - from: postgres:path.to.my_dataset name: my_dataset params: - pg_host: localhost - pg_port: 5432 - pg_db: my_database - pg_user: my_user - pg_pass_key: my_secret -``` - -### Data Store - -```yaml -datasets: - - from: spiceai:path.to.my_dataset - name: my_dataset - acceleration: - engine: postgres - params: - pg_host: localhost - pg_port: 5432 - pg_db: my_database - pg_user: my_user - pg_pass_key: my_secret + pg_host: localhost + pg_port: 5432 + pg_db: my_database + pg_user: my_user + pg_pass_key: my_secret ``` Additionally, an `engine_secret` may be provided when configuring a PostgreSQL data store to allow for using a different secret store to specify the password for a dataset using PostgreSQL as both the data source and data store. @@ -79,18 +47,18 @@ datasets: - from: spiceai:path.to.my_dataset name: my_dataset params: + pg_host: localhost + pg_port: 5432 + pg_db: data_store + pg_user: my_user + pg_pass_key: my_secret + acceleration: + engine: postgres + engine_secret: pg_backend + params: pg_host: localhost - pg_port: 5432 + pg_port: 5433 pg_db: data_store pg_user: my_user pg_pass_key: my_secret - acceleration: - engine: postgres - engine_secret: pg_backend - params: - pg_host: localhost - pg_port: 5433 - pg_db: data_store - pg_user: my_user - pg_pass_key: my_secret -``` \ No newline at end of file +``` diff --git a/spiceaidocs/content/en/Connectors/s3.md b/spiceaidocs/content/en/data-connectors/s3.md similarity index 70% rename from spiceaidocs/content/en/Connectors/s3.md rename to spiceaidocs/content/en/data-connectors/s3.md index b1f9f88b..2dd4e298 100644 --- a/spiceaidocs/content/en/Connectors/s3.md +++ b/spiceaidocs/content/en/data-connectors/s3.md @@ -1,41 +1,44 @@ --- type: docs -title: "S3 Data Connector" -linkTitle: "S3 Data Connector" -description: 'S3 Data Connector YAML reference' +title: 'S3 Data Connector' +linkTitle: 'S3 Data Connector' +description: 'S3 Data Connector Documentation' --- S3 as a connector for federated SQL query across Parquet files stored in S3, or S3-compatible storage solutions (e.g. Minio, Cloudflare R2). ## `params` -- `endpoint`: The S3 endpoint, or equivalent (e.g. Minio endpoint), for the S3-compatible storage. -- `region`: Region of the S3 bucket, if region specific. +- `endpoint`: The S3 endpoint, or equivalent (e.g. Minio endpoint), for the S3-compatible storage. +- `region`: Region of the S3 bucket, if region specific. ## `auth` -Check [Secrets]({{}}). +Check [Secrets Stores]({{}}). Required attribbutes: + - `key`: The access key authorised to access the S3 data (e.g. `AWS_ACCESS_KEY_ID` for AWS) - `secret`The secret key authorised to access the S3 data (e.g. `AWS_SECRET_ACCESS_KEY` for AWS) - ## Example + ### Minio + ```yaml - from: s3://s3-bucket-name/path/to/parquet/cool_dataset.parquet name: cool_dataset params: endpoint: https://my.minio.server - region: "us-east-1" # Best practice for Minio + region: 'us-east-1' # Best practice for Minio ``` #### S3 + ```yaml - from: s3://my-startups-data/path/to/parquet/cool_dataset.parquet name: cool_dataset params: endpoint: http://my-startups-data.s3.amazonaws.com - region: "ap-southeast-2" -``` \ No newline at end of file + region: 'ap-southeast-2' +``` diff --git a/spiceaidocs/content/en/data-ingestion/_index.md b/spiceaidocs/content/en/data-ingestion/_index.md new file mode 100644 index 00000000..1d083c2b --- /dev/null +++ b/spiceaidocs/content/en/data-ingestion/_index.md @@ -0,0 +1,13 @@ +--- +type: docs +title: 'Data Ingestion' +linkTitle: 'Data Ingestion' +description: '' +weight: 40 +--- + +Data can be ingested by the Spice runtime for replication to a Data Connector, like PostgreSQL or the Spice.ai Cloud platform. + +By default, the runtime exposes an [OpenTelemety](https://opentelemetry.io) (OTEL) endpoint at grpc://127.0.0.1:50052 for data ingestion. + +OTEL metrics will be inserted into datasets with matching names (metric name = dataset name) and optionally replicated to the dataset source. diff --git a/spiceaidocs/content/en/federated-queries/_index.md b/spiceaidocs/content/en/federated-queries/_index.md new file mode 100644 index 00000000..a6db40de --- /dev/null +++ b/spiceaidocs/content/en/federated-queries/_index.md @@ -0,0 +1,7 @@ +--- +type: docs +title: 'Federated Queries' +linkTitle: 'Federated Queries' +description: '' +weight: 20 +--- diff --git a/spiceaidocs/content/en/getting-started/_index.md b/spiceaidocs/content/en/getting-started/_index.md index c864268c..f3c8eb9c 100644 --- a/spiceaidocs/content/en/getting-started/_index.md +++ b/spiceaidocs/content/en/getting-started/_index.md @@ -1,16 +1,12 @@ --- type: docs -title: "Getting started with Spice.ai" -linkTitle: "Getting started" -weight: 20 -description: "Get started with Spice.ai in 5 minutes" +title: 'Getting started with Spice.ai' +linkTitle: 'Getting started' +weight: 10 +description: 'Get started with Spice.ai in 5 minutes' no_list: true --- -{{% alert title="Spice.ai Concepts" color="primary" %}} -Before following this guide it is first recommended to review the Spice.ai [core concepts](/). -{{% /alert %}} - ### Follow these steps to get started with Spice.ai. **Step 1.** Install the Spice CLI: diff --git a/spiceaidocs/content/en/local-acceleration/_index.md b/spiceaidocs/content/en/local-acceleration/_index.md new file mode 100644 index 00000000..b90ef264 --- /dev/null +++ b/spiceaidocs/content/en/local-acceleration/_index.md @@ -0,0 +1,7 @@ +--- +type: docs +title: 'Local Acceleration' +linkTitle: 'Local Acceleration' +description: '' +weight: 30 +--- diff --git a/spiceaidocs/content/en/machine-learning/_index.md b/spiceaidocs/content/en/machine-learning/_index.md new file mode 100644 index 00000000..600bf89b --- /dev/null +++ b/spiceaidocs/content/en/machine-learning/_index.md @@ -0,0 +1,7 @@ +--- +type: docs +title: 'Machine Learning' +linkTitle: 'Machine Learning' +description: '' +weight: 60 +--- diff --git a/spiceaidocs/content/en/machine-learning/inference/_index.md b/spiceaidocs/content/en/machine-learning/inference/_index.md new file mode 100644 index 00000000..1e95bed3 --- /dev/null +++ b/spiceaidocs/content/en/machine-learning/inference/_index.md @@ -0,0 +1,7 @@ +--- +type: docs +title: 'Machine Learning Inference' +linkTitle: 'Machine Learning Inference' +description: '' +weight: 20 +--- diff --git a/spiceaidocs/content/en/machine-learning/model-deployment/_index.md b/spiceaidocs/content/en/machine-learning/model-deployment/_index.md new file mode 100644 index 00000000..fa143cfd --- /dev/null +++ b/spiceaidocs/content/en/machine-learning/model-deployment/_index.md @@ -0,0 +1,7 @@ +--- +type: docs +title: 'ML Model Deployment' +linkTitle: 'ML Model Deployment' +description: '' +weight: 10 +--- diff --git a/spiceaidocs/content/en/reference/Spicepod/_index.md b/spiceaidocs/content/en/reference/Spicepod/_index.md index f1301f97..f7c25487 100644 --- a/spiceaidocs/content/en/reference/Spicepod/_index.md +++ b/spiceaidocs/content/en/reference/Spicepod/_index.md @@ -26,7 +26,7 @@ The name of the Spicepod. ## `secrets` -The secrets section in the Spicepod manifest is optional and is used to configure how secrets are stored and accessed by the Spicepod. [Learn more]({{}}). +The secrets section in the Spicepod manifest is optional and is used to configure how secrets are stored and accessed by the Spicepod. [Learn more]({{}}). ### `secrets.store` @@ -38,6 +38,7 @@ The type of secret store for reading secrets. - `keyring` Example + ```yaml secrets: store: env @@ -60,7 +61,7 @@ metadata: ## `datasets` -A Spicepod can contain one or more [datasets]({{}}) referenced by relative path. +A Spicepod can contain one or more [datasets]({{}}) referenced by relative path. **Example** diff --git a/spiceaidocs/content/en/reference/_index.md b/spiceaidocs/content/en/reference/_index.md index 09c6e60d..25c66f94 100644 --- a/spiceaidocs/content/en/reference/_index.md +++ b/spiceaidocs/content/en/reference/_index.md @@ -1,7 +1,7 @@ --- type: docs -title: "Spice.ai Reference Docs" -linkTitle: "Reference" -weight: 60 -description: "Reference documentation on the Spice.ai API, CLI and Pod manifest syntax." +title: 'Spice.ai Reference Docs' +linkTitle: 'Reference' +weight: 110 +description: 'Reference documentation on the Spice.ai API, CLI and Pod manifest syntax.' --- diff --git a/spiceaidocs/content/en/secrets/_index.md b/spiceaidocs/content/en/secret-stores/_index.md similarity index 87% rename from spiceaidocs/content/en/secrets/_index.md rename to spiceaidocs/content/en/secret-stores/_index.md index b1ae7df2..acf35ab6 100644 --- a/spiceaidocs/content/en/secrets/_index.md +++ b/spiceaidocs/content/en/secret-stores/_index.md @@ -1,18 +1,12 @@ --- type: docs -title: "Secrets" -linkTitle: "Secrets" +title: 'Secret Stores' +linkTitle: 'Secret Stores' description: '' -weight: 40 +weight: 90 --- -## Secret - -A Secret is an object used to store sensitive data, like a passwords, a tokens, or keys. - -## Secret Stores - -The secret store is a location where the `secret` objects are stored. +A secret store is a location where `secret` objects are stored, which are used to store sensitive data, like a passwords, a tokens, or keys. Spice.ai supports four types of secret stores: `file`, `env`, `kubernetes` and `keyring`. The type of secret store is specified in the `store` field of the `secrets` section in the Spicepod manifest. @@ -23,7 +17,7 @@ The Spice.ai CLI offers the `spice login` command to streamline credential stora ### Environment Secret Store -The `env` store type allows Spice.ai to read secrets from environment variables. The environment variables should be formatted like `SPICE_SECRET__`. +The `env` store type allows Spice.ai to read secrets from environment variables. The environment variables should be formatted like `SPICE_SECRET__`. All variables with the same prefix `SPICE_SECRET_` are combined into a single secret. This allows you to group related secret values under a single secret name. @@ -60,9 +54,9 @@ apiVersion: rbac.authorization.k8s.io/v1 metadata: name: spiced-account-role rules: - - apiGroups: [""] - resources: ["secrets"] - verbs: ["get"] + - apiGroups: [''] + resources: ['secrets'] + verbs: ['get'] ``` ### Keyring Secret Store @@ -93,6 +87,3 @@ Then set `store` field of the `secrets` section in the Spicepod manifest: secrets: store: keyring ``` - - -