Anybody tried creating a web application for live audio streaming + transcribing

[gvijqb]

I am looking for a simply jquery application that records audio, creates audio chunks of a couple of seconds, converts them to wav or mp3 format and sends them to a whisper server for transcription while recording next chunk of audio and then displaying the transcript of last chunk on screen.

So in a way it is live streaming transcription where I am speaking something and it is getting transcribed and displayed back in real time.

Any resources or guides on this?

[abhimishra01]

Subscribing to this Thread, as I would also like to get more insights on this from the community.
We did implement something similar, but the stream is not as real time as we expect ( similar to what we see in gmeet calls, where captions get generated in milliseconds)
And if the audio is in hindi + english mix, the transcription takes hell of a time which contradicts our use case for real time streaming
and further processing.
We tried with high end sagemaker notebook GPU based instances, EC2, p & m class instances, but yet the transcription can't happen within milliseconds.

[joramirez764]

Here's what they're looking for
https://github.com/yohasebe/whisper-stream

[makaveli10]

@abhimishra01 WhisperLive is real time and uses faster-whisper backend and has both firefox and chrome extension to transcribe in the browser. A webapp shouldn't be too hard with this backend.

[Gldkslfmsd]

Hi, I recommend Whisper-Streaming: #1978

It's also integrated in a web app https://github.com/ufal/correctable-lecture-translator . Ping the authors if you need documentation.