Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Hold] Platform UI: Add TogetherAI embedding provider and models #432

Draft
wants to merge 1 commit into
base: main
Choose a base branch
from
Draft

[Hold] Platform UI: Add TogetherAI embedding provider and models #432

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 13 additions & 3 deletions platform/embedding.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -63,10 +63,20 @@ To generate embeddings, choose one of the following embedding providers and mode

<Note>You can change a workflow's preconfigured provider only through [Custom](/platform/workflows#create-a-custom-workflow) workflow settings.</Note>

- **OpenAI**: Use [OpenAI](https://openai.com) to generate embeddings. Also, choose the model to use:
- **Azure OpenAI**: Use [Azure OpenAI](https://azure.microsoft.com/products/ai-services/openai-service) to generate embeddings. Also, choose the model to use:

- **text-embedding-3-small**, with 1536 dimensions.
- **text-embedding-3-large**, with 3072 dimensions.
- **Ada 002 (Text)**, with 1536 dimensions.
- **Ada 002 (Text)** (`text-embedding-ada-002`), with 1536 dimensions.

[Learn more](https://platform.openai.com/docs/guides/embeddings).
[Learn more](https://learn.microsoft.com/azure/ai-services/openai/concepts/models#embeddings).

- **TogetherAI**: Use [TogetherAI](https://www.together.ai/) to generate embeddings. Also, choose the model to use:

- **M2-BERT-80M-2K-Retrieval**, with 768 dimensions.
- **M2-BERT-80M-8K-Retrieval**, with 768 dimensions.
- **M2-BERT-80M-32K-Retrieval**, with 768 dimensions.
- **Sentence-BERT**, with 768 dimensions.
- **BERT**, with 768 dimensions.

[Learn more](https://docs.together.ai/docs/serverless-models#embedding-models).
32 changes: 26 additions & 6 deletions platform/workflows.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -290,13 +290,23 @@ There are two ways to create a custom workflow:
16. In the **Embed** area, for **Provider**, choose one of the following:

- **None**: Do not generate embeddings.
- **OpenAI**: Use OpenAI to generate embeddings. Also, choose the model to use:
- **Azure OpenAI**: Use Azure OpenAI to generate embeddings. Also, choose the model to use:

- **text-embedding-3-small**, with 1536 dimensions.
- **text-embedding-3-large**, with 3072 dimensions.
- **Ada 002 (Text)**, with 1536 dimensions.
- **Ada 002 (Text)** (`text-embedding-ada-002`), with 1536 dimensions.

[Learn more](https://platform.openai.com/docs/guides/embeddings).
[Learn more](https://learn.microsoft.com/azure/ai-services/openai/concepts/models#embeddings).

- **TogetherAI**: Use TogetherAI to generate embeddings. Also, choose the model to use:

- **M2-BERT-80M-2K-Retrieval**, with 768 dimensions.
- **M2-BERT-80M-8K-Retrieval**, with 768 dimensions.
- **M2-BERT-80M-32K-Retrieval**, with 768 dimensions.
- **Sentence-BERT**, with 768 dimensions.
- **BERT**, with 768 dimensions.

[Learn more](https://docs.together.ai/docs/serverless-models#embedding-models).

Learn more:

Expand Down Expand Up @@ -478,13 +488,23 @@ There are two ways to create a custom workflow:
<Accordion title="Embedder node">
For **Providers**, select one of the following:

- **OpenAI**: Use OpenAI to generate embeddings. Also, choose the model to use:
- **Azure OpenAI**: Use Azure OpenAI to generate embeddings. Also, choose the model to use:

- **text-embedding-3-small**, with 1536 dimensions.
- **text-embedding-3-large**, with 3072 dimensions.
- **Ada 002 (Text)**, with 1536 dimensions.
- **Ada 002 (Text)** (`text-embedding-ada-002`), with 1536 dimensions.

[Learn more](https://learn.microsoft.com/azure/ai-services/openai/concepts/models#embeddings).

- **TogetherAI**: Use TogetherAI to generate embeddings. Also, choose the model to use:

- **M2-BERT-80M-2K-Retrieval**, with 768 dimensions.
- **M2-BERT-80M-8K-Retrieval**, with 768 dimensions.
- **M2-BERT-80M-32K-Retrieval**, with 768 dimensions.
- **Sentence-BERT**, with 768 dimensions.
- **BERT**, with 768 dimensions.

[Learn more](https://platform.openai.com/docs/guides/embeddings).
[Learn more](https://docs.together.ai/docs/serverless-models#embedding-models).

Learn more:

Expand Down