please include openvino iGPU support for hardware acceleration

[jsapede]

as seen on frigate, openvino support highly accelerate ai computing (equivalent to google coral) even on old/small/cheap architectures (> gen6)
combined to faster-whisper or whisper.cpp could higly fluidify the experience of assistant.
iGPu can be easily passed to docker container as done on frigate.

[tannisroot]

Upstream doesn't support anything but CUDA unfortunately:
OpenNMT/CTranslate2#1072

[jsapede]

Totally Newb in thus domain, Dunno if it would bé helpful :

https://github.com/ggerganov/whisper.cpp/blob/master/models/convert-whisper-to-openvino.py

https://github.com/openvinotoolkit/openvino_notebooks/blob/main/notebooks/227-whisper-subtitles-generation/227-whisper-convert.ipynb

[tannisroot]

Totally Newb in thus domain, Dunno if it would bé helpful :

https://github.com/ggerganov/whisper.cpp/blob/master/models/convert-whisper-to-openvino.py

https://github.com/openvinotoolkit/openvino_notebooks/blob/main/notebooks/227-whisper-subtitles-generation/227-whisper-convert.ipynb

That's whisper.cpp, a different project.

[synesthesiam]

This is more applicable to this project: https://github.com/rhasspy/wyoming-whisper-cpp

[jsapede]

I'm too noob to understand the différence between Whisper/Whisper.cpp except it's a C version.
From what i see it could be possible to compile whisper.cpp with openVINO support and then serve it with Wyoming ?

[Donkey545]

For those interested in this thread, I made use of a fork of CTranslate2 to build a Wyoming Faster Whisper for ROCm container.

Check it out here if you are interested. I don't have much hardware to test with, so All I have tested is my APU.

This performs about 15x faster than CPU faster-whisper for me, and about 5x faster than the Whisper.cpp implementation that can be found here.