[Bug]: During streaming tokens should not be emitted if they could form a stop sequence

[yunfeng-scale]

Your current environment

The output of `python collect_env.py`

🐛 Describe the bug

Using mistral 7b, prompt Here is the English alphabet: ABC, temperature 0, stop sequence DEFGHIJ, during streaming, DEF, G, HI tokens are output (J triggers the stop). Expected behavior is no output. Tokens should be buffered when they could form prefix of any stop sequences, and sent out when that condition is cleared.

[njhill]

@yunfeng-scale I'm currently also already working on a fix for this as part of the same set of changes mentioned in #3572.