-
Notifications
You must be signed in to change notification settings - Fork 11
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Memory tooling and connector docs (#627)
* memory tooling and connector docs * restructuring of files * fix links
- Loading branch information
Showing
7 changed files
with
72 additions
and
2 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,26 @@ | ||
--- | ||
title: 'Memory Data Connector' | ||
sidebar_label: 'Memory Data Connector' | ||
description: 'Memory Data Connector Documentation' | ||
pagination_prev: null | ||
--- | ||
|
||
The Memory Data Connector enables configuring an in-memory dataset for tables used, or produced by the Spice runtime. Only certain tables, with predefined schemas, can be defined by the connector. These are: | ||
- `store`: Defines a table that LLMs, with [memory tooling](/features/large-language-models/memory), can store data in. Requires `mode: read_write`. | ||
|
||
### Examples | ||
|
||
```yaml | ||
datasets: | ||
- from: memory:store | ||
name: llm_memory | ||
mode: read_write | ||
columns: | ||
- name: value | ||
embeddings: # Easily make your LLM learnings searchable. | ||
- from: all-MiniLM-L6-v2 | ||
|
||
embeddings: | ||
- name: all-MiniLM-L6-v2 | ||
from: huggingface:huggingface.co/sentence-transformers/all-MiniLM-L6-v2 | ||
``` |
File renamed without changes.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,35 @@ | ||
--- | ||
title: 'Language Model Memories' | ||
sidebar_label: 'Memory' | ||
description: 'Learn how LLMs can interact with the spice runtime.' | ||
sidebar_position: 3 | ||
pagination_prev: null | ||
pagination_next: null | ||
--- | ||
|
||
# Memory Tools | ||
|
||
Spice provides memory persistence tools that allow language models to store and retrieve information across conversations. These tools are available through the `memory` tool group. | ||
|
||
## Enabling Memory Tools | ||
|
||
To enable memory tools for Spice models you need to: | ||
1. Define a `store` [memory](/components/data-connectors/memory.md) dataset. | ||
2. Specify `memory` in the model's `spice_tools` parameter. | ||
|
||
```yaml | ||
datasets: | ||
- from: memory:store | ||
name: llm_memory | ||
mode: read_write | ||
|
||
models: | ||
- name: memory-enabled-model | ||
from: openai:gpt-4o | ||
params: | ||
spice_tools: memory, sql # Can be combined with other tool groups | ||
``` | ||
## Available Tools | ||
- `store_memory`: Store important information for future reference | ||
- `load_memory`: Retrieve previously stored memories from the last time period. |
Empty file.
2 changes: 1 addition & 1 deletion
2
.../features/ai-gateway/default_overrides.md → ...ge-language-models/parameter_overrides.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
File renamed without changes.