StyleTTS/Kokoro

[ecyht2]

Model description

I think it would be nice to support StyleTTS or Kokoro-82M. Currently both of them isn't supported in transformers or optimum but there are onnx exports for StyleTTS and Kokoro-82M.

There are two main things that might cause problems:

1. Phonemizer
2. Saving .wav file

Prerequisites

• The model is supported in Transformers (i.e., listed here)
• The model can be exported to ONNX with Optimum (i.e., listed here)

Additional information

No response

Your contribution

I saw an implementation of inference using onnx-web here. Furthermore, I have exported the voice packs here.

[xenova]

Hi there 👋 The model is indeed supported now (#1148). You can use it with the kokoro-js library I created:

npm i kokoro-js

You can then generate speech as follows:

import { KokoroTTS } from "kokoro-js";

const model_id = "onnx-community/Kokoro-82M-ONNX";
const tts = await KokoroTTS.from_pretrained(model_id, {
  dtype: "q8", // Options: "fp32", "fp16", "q8", "q4", "q4f16"
});

const text = "Life is like a box of chocolates. You never know what you're gonna get.";
const audio = await tts.generate(text, {
  // Use `tts.list_voices()` to list all available voices
  voice: "af_bella",
});
audio.save("audio.wav");

See hexgrad/kokoro#3 for more information.