Adding warm up for Zipformer2

[manickavela29]

Hi @csukuangfj ,

How and Where to use?

use '--warmp-up=<#warmup> for running warm ups for zipformer model'
- Only supports Zipformer2 for now
- Based on requirement it can be extended

Why this is required?

1. It is a common practice to add warmup's for inference engines (especially with CUDA).
2. When the engine/server is live, first inference is very slow as it used for onnxruntime to create memory. Having warmup's will reduce the time.
3. Other execution providers of onnxruntime wlll also benefit from this.

Data points :

Without warmup :

online-websocket-server.cc:main:79 Started!
online-websocket-server.cc:main:80 Listening on: 6007
online-websocket-server.cc:main:81 Number of work threads: 8
online-zipformer2-transducer-model.cc:RunEncoder:444 Encoder Duration : 0.457923ms
online-zipformer2-transducer-model.cc:RunEncoder:444 Encoder Duration : 0.170431ms
online-zipformer2-transducer-model.cc:RunEncoder:444 Encoder Duration : 0.167805ms
online-zipformer2-transducer-model.cc:RunEncoder:444 Encoder Duration : 0.168272ms
online-zipformer2-transducer-model.cc:RunEncoder:444 Encoder Duration : 0.173339ms
online-zipformer2-transducer-model.cc:RunEncoder:444 Encoder Duration : 0.169113ms
online-zipformer2-transducer-model.cc:RunEncoder:444 Encoder Duration : 0.167528ms
online-zipformer2-transducer-model.cc:RunEncoder:444 Encoder Duration : 0.169950ms
online-zipformer2-transducer-model.cc:RunEncoder:444 Encoder Duration : 0.175659ms

With Warmup:

online-zipformer2-transducer-model.cc:RunEncoder:444 Encoder Duration : 0.524530ms
online-zipformer2-transducer-model.cc:RunEncoder:444 Encoder Duration : 0.190443ms
online-zipformer2-transducer-model.cc:RunEncoder:444 Encoder Duration : 0.188810ms
online-websocket-server-impl.cc:Run:250 Warm up completd : 3 times.
online-websocket-server.cc:main:79 Started!
online-websocket-server.cc:main:80 Listening on: 6007
online-websocket-server.cc:main:81 Number of work threads: 8
online-zipformer2-transducer-model.cc:RunEncoder:444 Encoder Duration : 0.193825ms
online-zipformer2-transducer-model.cc:RunEncoder:444 Encoder Duration : 0.169870ms
online-zipformer2-transducer-model.cc:RunEncoder:444 Encoder Duration : 0.169398ms
online-zipformer2-transducer-model.cc:RunEncoder:444 Encoder Duration : 0.172992ms
online-zipformer2-transducer-model.cc:RunEncoder:444 Encoder Duration : 0.172658ms
online-zipformer2-transducer-model.cc:RunEncoder:444 Encoder Duration : 0.173597ms
online-zipformer2-transducer-model.cc:RunEncoder:444 Encoder Duration : 0.173040ms
online-zipformer2-transducer-model.cc:RunEncoder:444 Encoder Duration : 0.171519ms
online-zipformer2-transducer-model.cc:RunEncoder:444 Encoder Duration : 0.172292ms

Let me know if additional change are required.
Thank you!

[csukuangfj]

Could you fix the build errors reported by the CI?

[csukuangfj]

For instance, see the following error from
https://github.com/k2-fsa/sherpa-onnx/actions/runs/8674782410/job/23787499410#step:12:1178

In file included from /home/runner/work/sherpa-onnx/sherpa-onnx/sherpa-onnx/csrc/online-recognizer-impl.cc:9:
/home/runner/work/sherpa-onnx/sherpa-onnx/sherpa-onnx/csrc/online-recognizer-transducer-impl.h: In member function ‘void sherpa_onnx::OnlineRecognizerTransducerImpl::WarmpUpRecognizer(size_t, int32_t) const’:
/home/runner/work/sherpa-onnx/sherpa-onnx/sherpa-onnx/csrc/online-recognizer-transducer-impl.h:190:13: error: expected primary-expression before ‘chunk_size’
  190 |     int32_t chunk_size = model_->ChunkSize();
      |             ^~~~~~~~~~
/home/runner/work/sherpa-onnx/sherpa-onnx/sherpa-onnx/csrc/online-recognizer-transducer-impl.h:190:12: error: expected ‘;’ before ‘chunk_size’
  190 |     int32_t chunk_size = model_->ChunkSize();
      |            ^~~~~~~~~~~
      |            ;
/home/runner/work/sherpa-onnx/sherpa-onnx/sherpa-onnx/csrc/online-recognizer-transducer-impl.h:190:13: error: ‘chunk_size’ was not declared in this scope
  190 |     int32_t chunk_size = model_->ChunkSize();
      |             ^~~~~~~~~~
At global scope:
cc1plus: note: unrecognized command-line option ‘-Wno-missing-template-keyword’ may have been intended to silence earlier diagnostics
make[2]: *** [sherpa-onnx/csrc/CMakeFiles/sherpa-onnx-core.dir/build.make:776: sherpa-onnx/csrc/CMakeFiles/sherpa-onnx-core.dir/online-recognizer-impl.cc.o] Error 1
make[2]: *** Waiting for unfinished jobs....

[manickavela29]

additional review : @pavankumar-ds

[csukuangfj]

Thanks!

Left some minor comments.

[csukuangfj]

Thank you for your contribution!

[manickavela29]

Thank you 😄