Fastest settings

[sailingbo]

I'm working towards an even faster implementation. I've tweaked the settings as best as I can to speed up the responses, but there are still times when (if I'm talking fast) I'll get chucks of 7-10 words transcribed at once. I want to get as close to 1-2 word chunks as possible. What settings would need to be tweaked to break up the sentences into shorter chunks?

[KoljaB]

Which platform, cpu type and gpu type are you using?

[sailingbo]

Windows 10, Intel i7-9800x, 3792mhz, 8 cores. WSL2 running Ubuntu. Nvidia Quadro RTX 4000.

[KoljaB]

Some things you might want to check: have CUDA enabled for your torch under WSL for the RTX. Set realtime_model_type to tiny.en. Set realtime_processing_pause to 0. default compute_type (fp16) will probably the fastest for your setup, so I don't think trying out int8 or float32 will be faster.

RealtimeSTT depends on faster-whisper, which offers some more tuning options. I can't take them all over into this API, it would get too huge, but if you want to dive deeper in, fork the code of this lib and look into the faster-whisper transcribe-calls in the code. You could for example try to lower the beam_size to 1 or 2. Not much more that I can think of currently but there are like 30 parameters.