s3 parquet doc (#111)

* s3 parquet doc * Update index * Jack/24 03 15 (#116) * Update acknowledgments (#109) * Update acknowledgments * Remove spiceai packages * update S3 connector docs * Revert "Update acknowledgments (#109)" This reverts commit 8b2c3e1. --------- Co-authored-by: Evgenii Khramkov <[email protected]> --------- Co-authored-by: Evgenii Khramkov <[email protected]> Co-authored-by: Jack Eadie <[email protected]>
spiceai · Mar 15, 2024 · 262082c · 262082c
1 parent 894ae5f
commit 262082c
Show file tree

Hide file tree

Showing 2 changed files with 51 additions and 0 deletions.
diff --git a/spiceaidocs/content/en/reference/Connectors/_index.md b/spiceaidocs/content/en/reference/Connectors/_index.md
@@ -0,0 +1,9 @@
+---
+type: docs
+title: "Data Connectors"
+linkTitle: "Data Connectors"
+description: ''
+weight: 70
+---
+
+
diff --git a/spiceaidocs/content/en/reference/Connectors/s3.md b/spiceaidocs/content/en/reference/Connectors/s3.md
@@ -0,0 +1,42 @@
+---
+type: docs
+title: "S3 Data Connector"
+linkTitle: "S3 Data Connector"
+description: 'S3 Data Connector YAML reference'
+weight: 80
+---
+
+S3 as a connector for federated SQL query across Parquet files stored in S3, or S3-compatible storage solutions (e.g. Minio, Cloudflare R2).
+
+## `params`
+
+- `endpoint`: The S3 endpoint, or equivalent (e.g. Minio endpoint), for the S3-compatible storage. 
+- `region`: Region of the S3 bucket, if region specific. 
+
+## `auth`
+
+Check [Secrets]({{<ref "reference/Secrets">}}).
+
+Required attribbutes:
+- `key`: The access key authorised to access the S3 data (e.g. `AWS_ACCESS_KEY_ID` for AWS)
+- `secret`The secret key authorised to access the S3 data (e.g. `AWS_SECRET_ACCESS_KEY` for AWS)
+
+
+## Example
+### Minio
+```yaml
+- from: s3://s3-bucket-name/path/to/parquet/cool_dataset.parquet
+  name: cool_dataset
+  params:
+    endpoint: https://my.minio.server
+    region: "us-east-1" # Best practice for Minio
+```
+
+#### S3
+```yaml
+- from: s3://my-startups-data/path/to/parquet/cool_dataset.parquet
+  name: cool_dataset
+  params:
+    endpoint: http://my-startups-data.s3.amazonaws.com
+    region: "ap-southeast-2"
+```