❓ Relative thresholding using VADIterator

[mukundt]

❓ Questions and Help

Hey! I've been really loving how accurate the VADIterator is for 8KHz telephony audio. Currently we have a static threshold of 0.85. This works well in most cases, but not in situations where (1) there is a large amount of background noise (2) there is a quiet environment but the speaker is also very quiet.

Do you have any suggestions for how to implement relative thresholds to deal with these cases? Thanks again!

[snakers4]

Hi,

Currently we have a static threshold of 0.85

One approach may be to fiddle with VAD activation and deactivation thresholds separately.
Another approach may be to actually have some accumulation buffer that postpones the VAD activation, but if activated, it activates from a previous early point.

(1) there is a large amount of background noise (2) there is a quiet environment but the speaker is also very quiet.

The VAD normalizes audio internally, so in a sense both of these cases are very similar.
As for the quiet audio, if all audio is quiet, you can try re-normalizing it and / or having some naive median rejection algorithm, where there is some median energy threshold where everything below this threshold is discarded as silence (or vice versa).

[mukundt]

Got it, thanks for the quick response!

The VAD normalizes audio internally, so in a sense both of these cases are very similar.

I might be doing something wrong, but it seems like the VADIterator is not normalizing audio as we are expecting. For example, see this graph of speech_probs. Visually (and aurally) it is very clear that the spikes correspond to speech, however the VAD was not triggered since it was below 0.85.

Here is the opposite example, where due to the large amount of background noise, the speech was continuously above 0.85.

When you mention re-normalizing the VAD or using a median rejection algorithm, could you give me some pointers on how I can implement that? Thank you again!

[snakers4]

is not normalizing audio as we are expecting.

The first chart looks like a proper VAD probability chart.

however the VAD was not triggered since it was below 0.85.

You can fiddle with both the activation and deactivation probability thresholds.
I believe they are hard-coded to be around 0.15 and 0.85.

When you mention re-normalizing the VAD or using a median rejection algorithm, could you give me some pointers on how I can implement that? Thank you again!

You can take a look at the get_speech_timestamps function here and improvise something for your streaming application.

I.e. collect some rolling statistics about loudness, collect some data about activation and only then decide, make min and max speech duration assumptions, etc etc