Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BUG] Input audio resampling is not efficiently implemented #28

Closed
mikejgray opened this issue Nov 7, 2023 · 1 comment · Fixed by #33
Closed

[BUG] Input audio resampling is not efficiently implemented #28

mikejgray opened this issue Nov 7, 2023 · 1 comment · Fixed by #33
Assignees
@NeonDaniel
Labels
bug Something isn't working

Comments

@mikejgray
Copy link
Contributor

Description

Input audio resampling is not efficiently implemented here. Consider pulling the implementation from ovos-stt-http-server or using helpers from neon_utils.file_utils

Steps to Reproduce

  • Record voice in Gradio interface
  • Press Submit to send to Neon
  • Wait 15-16+ seconds for the audio to convert and everything to process

Relevant Code

No response

Other Notes

No response
@mikejgray mikejgray added the bug Something isn't working label Nov 7, 2023
@NeonDaniel NeonDaniel self-assigned this Nov 9, 2023
@NeonDaniel
Copy link
Member

It may be better to skip the resampling here and add it to neon-speech so we can handle other clients that may not have sufficient processing power to do any conversion.

@NeonDaniel NeonDaniel mentioned this issue Nov 9, 2023
Merged
NeonDaniel added a commit to NeonGeckoCom/neon_speech that referenced this issue Nov 9, 2023
@NeonDaniel
Resample API input wav audio to ensure format matches listener config (…
778c7e4 
…#180)

# Description
Adds explicit handling of API input audio to set sample_rate,
sample_width, and channels to match listener configuration
Refactor to prevent creating an audio stream object not used for
non-streaming STT

# Issues
NeonGeckoCom/neon-iris#28

# Other Notes
<!-- Note any breaking changes, WIP changes, requests for input, etc.
here -->

Co-authored-by: Daniel McKnight <[email protected]>
NeonDaniel added a commit that referenced this issue Nov 9, 2023
@NeonDaniel
Remove audio resampling moved to neon-speech NeonGeckoCom/neon_speech…
dfb130a 
…#180

Closes #28
@NeonDaniel NeonDaniel mentioned this issue Nov 9, 2023
Merged
NeonDaniel added a commit that referenced this issue Nov 21, 2023
@NeonDaniel
Remove audio resampling and add timing context support (#33)
f14f71c 
* Remove audio resampling moved to neon-speech NeonGeckoCom/neon_speech#180
Closes #28

* Add timing context and logging to go with:
NeonGeckoCom/neon_speech#181
NeonGeckoCom/neon_audio#154

* More timing metrics and logging

* Refactor timing and add debug log
Prevent response error from affecting next input

---------

Co-authored-by: Daniel McKnight <[email protected]>
@github-actions github-actions bot mentioned this issue Dec 13, 2023
Merged
NeonDaniel added a commit that referenced this issue Dec 20, 2023
@NeonDaniel
0.1.0 (#49)
146481a 
# Changelog

## [0.0.5a21](https://github.com/NeonGeckoCom/neon-iris/tree/0.0.5a21)
(2023-12-13)

[Full
Changelog](0.0.5a20...0.0.5a21)

**Merged pull requests:**

- Update Documentation and Logging
[\#48](#48)
([NeonDaniel](https://github.com/NeonDaniel))

## [0.0.5a20](https://github.com/NeonGeckoCom/neon-iris/tree/0.0.5a20)
(2023-12-13)

[Full
Changelog](0.0.5a19...0.0.5a20)

**Merged pull requests:**

- Improved config handling
[\#46](#46)
([NeonDaniel](https://github.com/NeonDaniel))

## [0.0.5a19](https://github.com/NeonGeckoCom/neon-iris/tree/0.0.5a19)
(2023-12-11)

[Full
Changelog](0.0.5a18...0.0.5a19)

**Merged pull requests:**

- Update release automation
[\#47](#47)
([NeonDaniel](https://github.com/NeonDaniel))

## [0.0.5a18](https://github.com/NeonGeckoCom/neon-iris/tree/0.0.5a18)
(2023-12-11)

[Full
Changelog](0.0.4...0.0.5a18)

**Implemented enhancements:**

- \[FEAT\] Gradio User Settings Intents
[\#35](#35)
- \[FEAT\] After submitting recorded input audio the UI element should
reset [\#30](#30)
- \[FEAT\] Speech input should print STT results to the chat UI in
addition to response
[\#29](#29)

**Fixed bugs:**

- \[BUG\] Input audio resampling is not efficiently implemented
[\#28](#28)
- \[BUG\] Implement per-user settings in web UI
[\#27](#27)

**Merged pull requests:**

- Add GitHub pre-release automation
[\#45](#45)
([NeonDaniel](https://github.com/NeonDaniel))
- Update neon-mq-connector dependency to stable spec
[\#43](#43)
([NeonDaniel](https://github.com/NeonDaniel))
- Minor fixes to audio input handling
[\#42](#42)
([NeonDaniel](https://github.com/NeonDaniel))
- Refactor audio responses to utilize Chatbot UI
[\#40](#40)
([NeonDaniel](https://github.com/NeonDaniel))
- Get Language Support from Core
[\#37](#37)
([NeonDaniel](https://github.com/NeonDaniel))
- Handle profile updates
[\#36](#36)
([NeonDaniel](https://github.com/NeonDaniel))
- Handle STT Transcripts in Chat UI
[\#34](#34)
([NeonDaniel](https://github.com/NeonDaniel))
- Remove audio resampling and add timing context support
[\#33](#33)
([NeonDaniel](https://github.com/NeonDaniel))
- Fix web\_client language handling to respect configured input language
[\#32](#32)
([NeonDaniel](https://github.com/NeonDaniel))
- Threaded input handling and multi-session support
[\#31](#31)
([NeonDaniel](https://github.com/NeonDaniel))
- Add Gradio web UI with Docker Container
[\#24](#24)
([NeonDaniel](https://github.com/NeonDaniel))
- Add NeonVoiceClient class for minimal remote audio client
[\#23](#23)
([NeonDaniel](https://github.com/NeonDaniel))
- Resolve client compat. bug
[\#22](#22)
([NeonDaniel](https://github.com/NeonDaniel))
- Add CLI endpoints to interact with API and LLM endpoints
[\#21](#21)
([NeonDaniel](https://github.com/NeonDaniel))
- Specify `setup.py` path explicitly
[\#20](#20)
([NeonDaniel](https://github.com/NeonDaniel))
- Update build automation to default branch
[\#19](#19)
([NeonDaniel](https://github.com/NeonDaniel))
- Update to use shared version\_bump.py script
[\#18](#18)
([NeonDaniel](https://github.com/NeonDaniel))
- Remove invalid release Docker action
[\#17](#17)
([NeonDaniel](https://github.com/NeonDaniel))
- Update automation to shared repository
[\#15](#15)
([NeonDaniel](https://github.com/NeonDaniel))
- Add methods with CLI entrypoints to get STT/TTS
[\#12](#12)
([NeonDaniel](https://github.com/NeonDaniel))



\* *This Changelog was automatically generated by
[github_changelog_generator](https://github.com/github-changelog-generator/github-changelog-generator)*
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@NeonDaniel NeonDaniel

Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Remove audio resampling and add timing context support NeonGeckoCom/neon-iris
2 participants
@mikejgray @NeonDaniel