tiny diarize integration

[nichinungas]

Hey there,

What about adding tiny diarize to this also (which is now supported in whisper.cpp)? I have it working for pre-recorded but haven't quite figured out how to get it going with the live transcription. I was going to make what you've made here but looks like you've done a great job.

Nice work!

[Willxiam]

@nichinungas Diarization would be awesome. Vibe does not do live transcription yet correct only from a file. I would be curious how you are doing live transcription.

[thewh1teagle]

Vibe doesn't have yet live transcription but I'm planning to add it too,
The current way to do it is configure whisper to use small batches so it will update frequently and then simply register for callback.
I just need to check that it doesn't impact the performance.

[Willxiam]

@thewh1teagle I am thinking how live transcription would be great for notetaking. I could see this being outside the scope of vibe. But just in case I wanted to share some ideas.
-Timestamps at set in intervals
-User insertable notes or flags. Not sure how this would work, maybe in a separate pane that could be merged with the transcript if the user wanted.

There used to be some applications that would allow you to sync your live notes with a recording. Then after you could use your notes to navigate to the part of the recording you were listening when you made that note. That kind fo function would require the ability to record audio.

[thewh1teagle]

Sounds like useful feature.
I'm planning to add it,
I'm not sure if whisper.cpp supports label speakers, afaik it supports detecting speaker turns but not labels.

[deedeeharris]

With WhisperX I use wav2vec2 for speaker diarization. It may be implemented here as well.

[thewh1teagle]

Turns out that diarization feature in whisper.cpp is very limited. it works only in mono audio and it's detecting with very simple and unreliable strategy.
tinydiarize looks more promising but it support only english and the small model

[thewh1teagle]

Diarization added in latest version :)