Should the default length_penalty be 0?

[galatolofederico]

I'm wondering if the default value of length_penalty in the beam search algorithm should be set to 0.
My understanding, based on discussions like this issue in the hf transformers repository, is that a length_penalty greater than 0 favors shorter sequences, while a penalty less than 0 favors longer ones and 0 means no length_penalty

In the vLLM project (which has code adapted from Hugging Face), particularly in the file vllm/sequence.py, the default value is indeed 0:

def get_beam_search_score(self,
                          length_penalty: float = 0.0,
                          seq_len: Optional[int] = None,
                          eos_token_id: Optional[int] = None) -> float:
    """Calculate the beam search score with length penalty.
    Adapted from: https://github.com/huggingface/transformers/blob/ccb92be23def445f2afdea94c31286f84b89eb5b/src/transformers/generation/beam_search.py#L938
    """

Should we consider setting the default length_penalty to 0?

[hmellor]

The default length_penalty in get_beam_search_score was changed to 1.0 last week in #2667. So that it matches the one in SamplingParams.

This is the default because it's the default in transformers and the argument for this being the default in transformers is in huggingface/transformers#19006 (comment).