Repetitions and Hallucinations when using prompt feature

[vchagari]

Hi,

I see a lot of repetitions and hallucinations in the output when i use prompt feature, could you any one figured how to overcome this behaviour?, please let me know.

[JH90iOS]

same , the prompt is helpful when transcribe some technical terms ,but will cause specific hallucinations and repetitions . Is there any way to avoid this problem ? Thanks a lot !

[rahulbansal16]

I am also experiencing them. How is deepgram able to handle it?

[rahulbansal16]

@vchagari @JH90iOS did you find any solution?

[toanhuynhnguyen]

I face the same issue, anyone can help? Thanks.

[blackpolarz]

Not sure about what others did but what I did was to postprocess the output with the following methods.

1. Check the output of each sentence, filter out common hallucinations terms. You can refer to hallucination discussion where users have pointed out some of the common hallucinated words such as subtitles, some website links and more.
2. Split the sentences, record them and compare them with the next sentence to determine if they are repeated.
3. As for prompts being shown as hallucination, refer to 1 with slight modification such that it only filters the entire prompt.

These methods would reduce some hallucination.
If you want better removal, you might want to use LLM to filter each output but that would be extremely computationally expensive.

[toanhuynhnguyen]

Thanks friend. But we need to fine-tune Whisper parameters for this issue.