Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: add OpenAI Realtime API #28

Merged
merged 2 commits into from
Nov 13, 2024
Merged

docs: add OpenAI Realtime API #28

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
ce62e94
add realtime api docs
bryantanderson Nov 11, 2024
89536d0
nits
bryantanderson Nov 11, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions fern/docs.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -242,6 +242,8 @@ navigation:
path: tools-calling.mdx
- page: Prompting Guide
path: prompting-guide.mdx
- page: OpenAI Realtime
path: openai-realtime.mdx
- section: Glossary
contents:
- page: Definitions
Expand Down
16 changes: 16 additions & 0 deletions fern/openai-realtime.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
---
title: OpenAI Realtime
subtitle: You can use OpenAI's newest speech-to-speech model with your Vapi assistants.
slug: openai-realtime
---

<Note>
The Realtime API is currently in beta, and not recommended for production use by OpenAI. We're excited to have you try this new feature and welcome your [feedback](https://discord.com/invite/pUFNcf2WmH) as we continue to refine and improve the experience.
</Note>

OpenAI’s Realtime API enables developers to use a native speech-to-speech model. Unlike other Vapi configurations which orchestrate a transcriber, model and voice API to simulate speech-to-speech, OpenAI’s Realtime API natively processes audio in and audio out.

To start using it with your Vapi assistants, select `gpt-4o-realtime-preview-2024-10-01` as your model.
- Please note that only OpenAI voices may be selected while using this model. The voice selection will not act as a TTS (text-to-speech) model, but rather as the voice used within the speech-to-speech model.
- Also note that we don’t currently support Knowledge Bases with the Realtime API.
- Lastly, note that our Realtime integration still retains the rest of Vapi's orchestration layer such as the endpointing and interruption models to enable a reliable conversational flow.
Loading