Whisper improments API published?

[SinanAkkoyun]

Hi! Will all improvements made for the Whisper API be also published here?

[jongwook]

Hi, Whisper API currently supports a subset of what is possible with the open-source version in this repo. Whisper API's serving stack isn't planned to be open-sourced as it depends on the rest of the API platform, but I'd be open to PRs for optimized inference implementations, e.g. via TorchDynamo (as in #43) in the upcoming PyTorch 2.0.

[SinanAkkoyun]

Hi, thanks for the answer. I see that they wouldn't publish the serving stack. But what you are also saying is that they optimize whisper on their own without publishing anything here, did I understand that right?

[jongwook]

Yes, currently we are serving the same large-v2 model in the API. The API is using inference code (as part of the serving stack) that is faster than the pure PyTorch implementation in this repository, but it's doing mathematically the same calculation.