Skip to content

Commit

Permalink
Add models spec documentation (#127)
Browse files Browse the repository at this point in the history
* Add models spec documentation

* add note

* add note

* update model warnings
  • Loading branch information
ewgenius authored Mar 25, 2024
1 parent 0ab3161 commit 436bdec
Show file tree
Hide file tree
Showing 5 changed files with 120 additions and 2 deletions.
2 changes: 2 additions & 0 deletions spiceaidocs/content/en/machine-learning/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,3 +5,5 @@ linkTitle: 'Machine Learning'
description: ''
weight: 60
---

{{< alert color="warning" title="Early Preview" >}}The Spice ML runtime is in its early preview phase and is subject to modifications.{{< /alert >}}
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
---
type: docs
title: "Huggingface"
linkTitle: "Huggingface"
weight: 100
---
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
---
type: docs
title: "SpiceAI"
linkTitle: "SpiceAI"
weight: 90
---
14 changes: 12 additions & 2 deletions spiceaidocs/content/en/reference/Spicepod/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -94,15 +94,25 @@ datasets:

## `models`

A Spicepod can contain one or more [models](https://docs.spice.ai/reference/specifications/models-yaml-specification) referenced by relative path.
A Spicepod can contain one or more [models]({{<ref "reference/spicepod/models">}}) referenced by relative path.

**Example**

A model referenced by path.

```yaml
models:
- from: models/gas_fees
- from: models/drive_stats
```

A model defined inline.

```yaml
models:
- from: spiceai:spice.ai/lukekim/smart/models/drive_stats:latest
name: drive_stats
datasets:
- drive_stats_inferencing
```

## `dependencies`
Expand Down
94 changes: 94 additions & 0 deletions spiceaidocs/content/en/reference/Spicepod/models.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,94 @@
---
type: docs
title: "Models"
linkTitle: "Models"
description: 'Models YAML reference'
weight: 90
---

{{< alert color="warning" title="Early Preview" >}}The model specifications are in early preview and are subject to change.{{< /alert >}}

A Spicepod can contain one or more models referenced by relative path, or defined inline.

# `models`

Inline example:

`spicepod.yaml`
```yaml
models:
- from: spiceai:spice.ai/lukekim/smart/models/drive_stats:latest
name: drive_stats
datasets:
- drive_stats_inferencing
```
`spicepod.yaml`
```yaml
models:
- from: huggingface:huggingface.co/spiceai/darts:latest
name: drive_stats
files:
- model.onnx
datasets:
- drive_stats_inferencing
```

Relative path example:

`spicepod.yaml`
```yaml
models:
- from: models/drive_stats
```

`models/drive_stats/model.yaml`
```yaml
models:
- from: spiceai:spice.ai/lukekim/smart/models/drive_stats:latest
name: drive_stats
datasets:
- drive_stats_inferencing
```

## `name`

The name of the model. This is used to reference the model in the pod manifest, as well as in external data sources.

## `from`

The `from` field is a string that represents the Uniform Resource Identifier (URI) for the model. This URI is composed of two parts: a prefix indicating the source of the model, and the actual link to the model.

The syntax for the `from` field is as follows:

```yaml
from: <source>:<link>:<version>
```

Where:

- `<source>`: The source of the model

Currently supported sources:
- [`spiceai`]({{<ref "machine-learning/model-deployment/spiceai">}})
- [`huggingface`]({{<ref "machine-learning/model-deployment/huggingface">}})

- `<link>`: The actual link to the model.

- `<version>`: The version of the model. This is optional and if not specified, the latest version of the model will be used.

## `datasets`

An array of dataset names needed for the model.

## `files`

Only for Huggingface models. List of model files to download from Huggingface.

```yaml
files:
- model.onnx
- model.onnx.data
...
```

0 comments on commit 436bdec

Please sign in to comment.