Min_tokens

[rlasseri]

Hi ! Thanks for your work,
A quick question is it possible to have a min_token parameters ?
Thanks

[zhuohan123]

I believe this is possible. You should be able to force the generation not to finish in this function:

vllm/vllm/engine/llm_engine.py

Line 419 in 79af7e9

def _stop_sequences(self, seq_groups: List[SequenceGroup]) -> None:

It would be great if you can contribute this feature!

[Lvjinhong]

So, has it been implemented now?

[hmellor]

You could use a negative length penalty

vllm/vllm/sampling_params.py

Lines 63 to 64 in 925f333

	length_penalty: Float that penalizes sequences based on their length.
	Used in beam search.

[njhill]

This has now been implemented and will be in the next version: #3124