Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add use-cases and components sections #314

Merged
merged 12 commits into from
Jul 11, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion spiceaidocs/docs/api/arrow-flight-sql/index.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
title: 'Arrow Flight SQL'
title: 'Arrow Flight SQL API'
sidebar_label: 'Arrow Flight SQL'
sidebar_position: 2
description: 'Query Spice using JDBC/ODBC/ADBC'
Expand Down
28 changes: 28 additions & 0 deletions spiceaidocs/docs/api/http/chat-completions.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
---
title: 'POST /v1/chat/completions'
sidebar_label: 'POST /v1/chat/completions'
description: ''
sidebar_position: 6
---

Chat completions is an OpenAI compatible endpoint.

Specify the model by providing the component name in the `model` key. For example:

```shell
curl http://localhost:3000/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{
"model": "my_language_model",
"messages": [
{
"role": "system",
"content": "I am just like any other OpenAI server!"
},
{
"role": "user",
"content": "That\'s cool!"
}
]
}'
```
40 changes: 40 additions & 0 deletions spiceaidocs/docs/api/http/datasets.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
---
title: 'GET /v1/datasets'
sidebar_label: 'GET /v1/datasets'
description: 'Fetch datasets'
sidebar_position: 2
---

Returns a the list of configured datasets.

Example:

```bash
curl --request GET \
--url http://localhost:3000/v1/datasets
```

Response:

```json
[
{
"from": "postgres:syncs",
"name": "daily_journal_accelerated",
"replication_enabled": false,
"acceleration_enabled": true
},
{
"from": "databricks:hive_metastore.default.messages",
"name": "messages_accelerated",
"replication_enabled": false,
"acceleration_enabled": true
},
{
"from": "postgres:aidemo_messages",
"name": "general",
"replication_enabled": false,
"acceleration_enabled": false
}
]
```
20 changes: 20 additions & 0 deletions spiceaidocs/docs/api/http/embeddings.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
---
title: 'POST /v1/embeddings'
sidebar_label: 'POST /v1/embeddings'
description: ''
sidebar_position: 7
---

Chat completions is an OpenAI compatible endpoint.

Specify the embedding model by providing the component name in the `model` key. For example:

```shell
curl http://localhost:3000/v1/embeddings \
-H "Content-Type: application/json" \
-d '{
"input": "The food was delicious and the waiter...",
"model": "text-embedding-ada-002",
"encoding_format": "float"
}'
```
89 changes: 5 additions & 84 deletions spiceaidocs/docs/api/http/index.md
Original file line number Diff line number Diff line change
@@ -1,90 +1,11 @@
---
title: 'HTTP API'
sidebar_label: 'HTTP'
sidebar_position: 1
description: 'Spice runtime HTTP API'
description: ''
pagination_prev: null
pagination_next: null
---

import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';
import DocCardList from '@theme/DocCardList';

The Spice runtime supports SQL queries directly from HTTP requests.

An example CuRL

```shell
curl -XPOST "127.0.0.1:3000/v1/sql" \
--data "SELECT avg(total_amount), \
avg(tip_amount), \
count(1), \
passenger_count \
FROM my_table \
GROUP BY passenger_count \
ORDER BY passenger_count ASC \
LIMIT 3"
```

And response

```json
[
{
"AVG(my_table.tip_amount)": 3.072259971396793,
"AVG(my_table.total_amount)": 25.327816939456525,
"COUNT(Int64(1))": 31465,
"passenger_count": 0
},
{
"AVG(my_table.tip_amount)": 3.3712622884680057,
"AVG(my_table.total_amount)": 26.205230445474996,
"COUNT(Int64(1))": 2188739,
"passenger_count": 1
},
{
"AVG(my_table.tip_amount)": 3.7171302113290854,
"AVG(my_table.total_amount)": 29.520659930930304,
"COUNT(Int64(1))": 405103,
"passenger_count": 2
}
]
```

This allows for simple integration into any language or framework
<Tabs>
<TabItem value="python" label="Python" default>

```python
from typing import Optional
import requests
import pandas as pd

def query_runtime(query: str, url: str = "http://127.0.0.1:3000") -> Optional[pd.DataFrame]:
response = requests.post(url, data=query)

if response.status_code != 200:
print(f"Error: Received status code {response.status_code}")
return None

return pd.DataFrame(response.json())
```

</TabItem>
<TabItem value="javascript" label="Javascript">
```javascript
async function queryRuntime(query, url = "http://127.0.0.1:3000") {
try {
const response = await fetch(url, {method: 'POST', body: query});
if (!response.ok) {
console.error(`Error: Received status code ${response.status}`);
return null;
}

return await response.json();
} catch (error) {
return error;
}
}
```

</TabItem>
</Tabs>
<DocCardList />
Original file line number Diff line number Diff line change
@@ -1,101 +1,13 @@
---
title: 'Model HTTP APIs'
sidebar_label: 'Model HTTP APIs'
title: POST /v1/predict'
sidebar_label: 'POST /v1/predict'
description: ''
sidebar_position: 2
pagination_prev: null
pagination_next: null
sidebar_position: 10
---

import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';

### GET `/v1/models`

Returns all currently available models

Parameters:
- `format`: The return payload format `'json'` or `'csv'`. Default `'json'`.


Response:
For each model, returns the spicepod component's `name`, `from` and `datasets`. For example
```json
[
{
"name": "rf_forecast",
"from": "file://usr/me/model.onnx",
"datasets": [
"input_times"
]
},
{
"name": "my_language_model",
"from": "openai:gpt-3.5-turbo",
"datasets": []
}
]

```

## Machine Learning
Spice includes dedicated predictions APIs.

### GET `/v1/models/:name/predict`

Make a prediction using a specific [deployed ML model](../ml-models/index.md).

Example:

```shell
curl "http://localhost:3000/v1/models/my_model_name/predict"
```

Parameters:

- `name`: References the model name defined in the `spicepod.yaml`.

#### Response

<Tabs>
<TabItem value="Success" label="Success" default>
```json
{
"status": "Success",
"model_name": "my_model_name",
"model_version": "1.0",
"lookback": 30,
"prediction": [0.45, 0.50, 0.55],
"duration_ms": 123
}
```
</TabItem>
<TabItem value="Bad Request" label="Bad Request">
```json
{
"status": "BadRequest",
"error_message": "You have me a bad request :(",
"model_name": "my_model_name",
"lookback": 30,
"duration_ms": 12
}
```
</TabItem>
<TabItem value="Internal Error" label="Internal Error">
```json
{
"status": "InternalError",
"error_message": "Oops, the server couldn't predict",
"model_name": "my_model_name",
"lookback": 30,
"duration_ms": 12
}
```
</TabItem>
</Tabs>

### POST `/v1/predict`

Make predictions using all loaded forecasting models in parallel, useful for ensembling or A/B testing.

Example:
Expand Down Expand Up @@ -152,37 +64,55 @@ Parameters:

:::

## Language Models
Spice supports OpenAI compatible endpoints
## GET `/v1/models/:name/predict`

Make a prediction using a specific [model](/components/models/index.md).

Example:

### POST `/v1/chat/completions`
To specify the model, provide the component name in the `model` key. For example
```shell
curl http://localhost:3000/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{
"model": "my_language_model",
"messages": [
{
"role": "system",
"content": "I am just like any other OpenAI server!"
},
{
"role": "user",
"content": "That\'s cool!"
}
]
}'
curl "http://localhost:3000/v1/models/my_model_name/predict"
```

### POST `/v1/embeddings`
To specify the embedding model, provide the component name in the `model` key. For example
```shell
curl http://localhost:3000/v1/embeddings \
-H "Content-Type: application/json" \
-d '{
"input": "The food was delicious and the waiter...",
"model": "text-embedding-ada-002",
"encoding_format": "float"
}'
```
Parameters:

- `name`: References the model name defined in the `spicepod.yaml`.

### Response

<Tabs>
<TabItem value="Success" label="Success" default>
```json
{
"status": "Success",
"model_name": "my_model_name",
"model_version": "1.0",
"lookback": 30,
"prediction": [0.45, 0.50, 0.55],
"duration_ms": 123
}
```
</TabItem>
<TabItem value="Bad Request" label="Bad Request">
```json
{
"status": "BadRequest",
"error_message": "You have me a bad request :(",
"model_name": "my_model_name",
"lookback": 30,
"duration_ms": 12
}
```
</TabItem>
<TabItem value="Internal Error" label="Internal Error">
```json
{
"status": "InternalError",
"error_message": "Oops, the server couldn't predict",
"model_name": "my_model_name",
"lookback": 30,
"duration_ms": 12
}
```
</TabItem>
</Tabs>
Loading