Update KITT and other plugins to use end_of_speech field #153
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Tested with KITT. It significantly improves the end of speech behavior so that we are giving it a 1s wait before starting to process user input.
|
We're building your pull request over on Zeet. |
askable-nic
added a commit
to askableorg/aimod-poc-livekit-agent
that referenced
this pull request
Feb 8, 2024
commit 5fab872 Author: Neil Dwyer <[email protected]> Date: Tue Feb 6 17:18:48 2024 -0800 Kitt to handle empty deepgram text + elevenlabs bug fix (livekit#155) hotfix commit 7661824 Author: Neil Dwyer <[email protected]> Date: Tue Feb 6 16:25:35 2024 -0800 Bump deepgram version (livekit#154) commit ddc168e Author: Neil Dwyer <[email protected]> Date: Tue Feb 6 16:10:52 2024 -0800 Fixes to Elevenlabs (livekit#152) commit 5555526 Author: David Zhao <[email protected]> Date: Tue Feb 6 09:12:38 2024 -0800 Update KITT and other plugins to use end_of_speech field (livekit#153) * Update KITT and other plugins to use end_of_speech field Tested with KITT. It significantly improves the end of speech behavior so that we are giving it a 1s wait before starting to process user input. * ruff on 3.10 * use ruff action * fixed ruff commit a98a7c1 Author: Paul Lockett <[email protected]> Date: Tue Feb 6 00:02:51 2024 -0800 update tts init to export StreamAdapter (livekit#149) commit 604d7e3 Author: Sean Muirhead <[email protected]> Date: Mon Feb 5 21:35:55 2024 -0800 Update deepgram endpointing (livekit#145) * deepgram: Add min_silence_duration to deepgram client. deepgram controls vad by endpointing parameter, this fix allows to configure min_silence_duration in agents layer * add utterance_end_ms and speech_final to Deepgram plugin * add utterance_end_ms and speech_final to Deepgram plugin * expose speech_final as end_of_speech --------- Co-authored-by: Lam Nguyen <[email protected]>
askable-nic
added a commit
to askableorg/aimod-poc-livekit-agent
that referenced
this pull request
Feb 22, 2024
commit 5fab872 Author: Neil Dwyer <[email protected]> Date: Tue Feb 6 17:18:48 2024 -0800 Kitt to handle empty deepgram text + elevenlabs bug fix (livekit#155) hotfix commit 7661824 Author: Neil Dwyer <[email protected]> Date: Tue Feb 6 16:25:35 2024 -0800 Bump deepgram version (livekit#154) commit ddc168e Author: Neil Dwyer <[email protected]> Date: Tue Feb 6 16:10:52 2024 -0800 Fixes to Elevenlabs (livekit#152) commit 5555526 Author: David Zhao <[email protected]> Date: Tue Feb 6 09:12:38 2024 -0800 Update KITT and other plugins to use end_of_speech field (livekit#153) * Update KITT and other plugins to use end_of_speech field Tested with KITT. It significantly improves the end of speech behavior so that we are giving it a 1s wait before starting to process user input. * ruff on 3.10 * use ruff action * fixed ruff commit a98a7c1 Author: Paul Lockett <[email protected]> Date: Tue Feb 6 00:02:51 2024 -0800 update tts init to export StreamAdapter (livekit#149) commit 604d7e3 Author: Sean Muirhead <[email protected]> Date: Mon Feb 5 21:35:55 2024 -0800 Update deepgram endpointing (livekit#145) * deepgram: Add min_silence_duration to deepgram client. deepgram controls vad by endpointing parameter, this fix allows to configure min_silence_duration in agents layer * add utterance_end_ms and speech_final to Deepgram plugin * add utterance_end_ms and speech_final to Deepgram plugin * expose speech_final as end_of_speech --------- Co-authored-by: Lam Nguyen <[email protected]>
parshvadaftari
pushed a commit
to parshvadaftari/agents
that referenced
this pull request
Jun 23, 2024
* Avoid division by zero
I saw the following in one of my tests:
```
Traceback (most recent call last):
File "/Users/shahafabileah/work/vocode/vocode-python/vocode/streaming/utils/worker.py", line 167, in _run_loop
await self.current_task
File "/Users/shahafabileah/work/vocode/vocode-python/vocode/streaming/streaming_conversation.py", line 284, in process
message_sent, cut_off = await self.conversation.send_speech_to_output(
File "/Users/shahafabileah/work/vocode/vocode-python/vocode/streaming/streaming_conversation.py", line 515, in send_speech_to_output
message_sent = f"{synthesis_result.get_message_up_to(seconds)}-"
File "/Users/shahafabileah/work/vocode/vocode-python/vocode/streaming/synthesizer/base_synthesizer.py", line 212, in <lambda>
lambda seconds: self.get_message_cutoff_from_total_response_length(
File "/Users/shahafabileah/work/vocode/vocode-python/vocode/streaming/synthesizer/base_synthesizer.py", line 161, in get_message_cutoff_from_total_response_length
estimated_output_seconds_per_char = estimated_output_seconds / len(message.text)
ZeroDivisionError: float division by zero
```
I haven't dug deeper to understand why the message is empty, and I don't know if 1 is a good fallback value.
* return string
---------
Co-authored-by: Kian <[email protected]>
SuJingnan
pushed a commit
to SuJingnan/agents
that referenced
this pull request
Nov 26, 2024
* Update KITT and other plugins to use end_of_speech field Tested with KITT. It significantly improves the end of speech behavior so that we are giving it a 1s wait before starting to process user input. * ruff on 3.10 * use ruff action * fixed ruff
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Tested with KITT. It significantly improves the end of speech behavior so that we are giving it a 1s wait before starting to process user input.