Mismatch in fbank computation parameters causing deletion errors

[ipmedenn]

Hello,

I found that there is a mismatch in fbank computation parameters.
Icefall models are trained with parameter high_freq = -400 (class FbankConfig in lhotse/features/kaldi/extractors.py), but in kaldi-native-fbank(sherpa-onnx) default high_freq value is 0.
In my experiments, for some audio files this causes a large amount of deletion errors in sherpa-onnx.
Also, similar mismatch in kaldifeat affects zipformer/streaming_decode.py in icefall.

[csukuangfj]

Thanks for pointing it out! We have never realized the mismatch.

I just checked the code and find that lhotse is using -400 as the default value for high_freq.

Could you help make PRs to sherpa to hardcode the value of high_freq to -400?

[nshmyrev]

Hi @ipmedenn

This is a great catch, thank you! I am actually fighting for some time with high number of deletions in streaming longform recognition too.

Do you see that the proposed change completely fixed the problem? I quickly tested with -400, actually I don't see much difference, it sometimes gets even worse. Maybe problem is somewhere else too. Do you see the deletions issue in offline mode or in streaming mode?

One thing I noticed that it helps to reset encoder state, but not well tested

alphacep@d6ae830

[csukuangfj]

Fixed in

• Use high_freq -400 in computing fbank features. #515
• Use high_freq -400 in computing fbank features. sherpa-ncnn#294
• Use high_freq -400 in computing fbank features. sherpa#528
• Use high_freq -400 in computing fbank features. icefall#1447

Many thanks to @ipmedenn !

[ipmedenn]

Hi @ipmedenn

This is a great catch, thank you! I am actually fighting for some time with high number of deletions in streaming longform recognition too.

Do you see that the proposed change completely fixed the problem? I quickly tested with -400, actually I don't see much difference, it sometimes gets even worse. Maybe problem is somewhere else too. Do you see the deletions issue in offline mode or in streaming mode?

One thing I noticed that it helps to reset encoder state, but not well tested

Hi @nshmyrev,
I see deletions issue in streaming mode, and -400 completely solves this issue.

[nshmyrev]

@ipmedenn Great, thanks for the information!