[Misc] Include matched stop string/token in responses

[njhill]

Currently a finish_reason of "stop" is returned if any of the following are encountered:

• One of the provided stop strings
• One of the provided stop tokens
• The EOS token

It can be useful to know specifically which of these caused the sequence generation to stop, especially since by default the stop strings/tokens are omitted from the output text (and output token_ids?).

This PR adds a stop_reason field to the CompletionOutput class which will contain the matched stop string or integer token id. It will be None otherwise, including the EOS token case. This means in particular that EOS can be inferred iff finish_reason=="stop" and stop_reason=None.

I've also added to the openai server responses but not sure whether or not this should be included since it isn't part of the official API.

Thanks @sahilsuneja1 for adding a test

[njhill]

@simon-mo would be good to get your thoughts on this simple addition. It would be useful for us even if it's not exposed in the openai API responses. Also not sure what the best field name is, perhaps stop_match instead of stop_reason?

[simon-mo]

I think this is very simple and useful! Naming it as stop_reason is fine, or maybe finish_reason_stop_object: {"kind": "eos"|"stop_string"|"stop_token_id":, "value": None| str | int}

[njhill]

Thanks @simon-mo!

Naming it as stop_reason is fine, or maybe finish_reason_stop_object: {"kind": "eos"|"stop_string"|"stop_token_id":, "value": None| str | int}

Will defer to your judgment on this! I like the simplicity of a single field but agree that more explicit could be better.

[simon-mo]

What you have is totally fine as long as it's well documented.

[njhill]

Thanks @simon-mo, I just pushed a small update to add descriptions to the new stop_reason openai response fields in protocol.py, and this is already documented in the CompletionOutput docstring. Is there anywhere else you think we should add some doc for this?