Skip to content

Commit

Permalink
Update datasets.md
Browse files Browse the repository at this point in the history
  • Loading branch information
phillipleblanc authored Jan 31, 2024
1 parent 81179a8 commit 889568e
Showing 1 changed file with 40 additions and 6 deletions.
46 changes: 40 additions & 6 deletions spiceaidocs/content/en/reference/Spicepod/datasets.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,13 +15,47 @@ Inline example:
`spicepod.yaml`
```yaml
datasets:
- name: spiceai.uniswap_v2_eth_usdc
type: overwrite
source: spice.ai
auth: spice.ai
- from: spice.ai/eth/beacon/eigenlayer
name: strategy_manager_deposits
params:
app: goerli-app
acceleration:
enabled: true
refresh: 1h
mode: inmemory # / file
engine: arrow # / duckdb
refresh_interval: 1h
refresh_mode: full / append # update / incremental
retention: 30m
```
`spicepod.yaml`
```yaml
datasets:
- from: databricks.com/spiceai/datasets
name: uniswap_eth_usd
params:
environment: prod
acceleration:
enabled: true
mode: inmemory # / file
engine: arrow # / duckdb
refresh_interval: 1h
refresh_mode: full / append # update / incremental
retention: 30m
```

`spicepod.yaml`
```yaml
datasets:
- from: local/Users/phillip/data/test.parquet
name: test
acceleration:
enabled: true
mode: inmemory # / file
engine: arrow # / duckdb
refresh_interval: 1h
refresh_mode: full / append # update / incremental
retention: 30m
```

Relative path example:
Expand Down Expand Up @@ -88,4 +122,4 @@ i.e. `1h` for 1 hour, `1m` for 1 minute, `1s` for 1 second, etc.

Optional. Only supported for `append` datasets. Specifies how long to retain data updates from the data source before they are deleted. Specified as a [duration literal]({{<ref "reference/duration">}}).

If not specified, the default retention is to keep all data.
If not specified, the default retention is to keep all data.

0 comments on commit 889568e

Please sign in to comment.