Add o1 Models

[Ryan526]

Add o1 models and change max_tokens to max_completion_tokens since the former is now deprecated.

Streaming can't be enabled in profiles using these models.

Max Output tokens for these new models:
o1-preview: Up to 32,768 tokens
o1-mini: Up to 65,536 tokens

[Ryan526]

I've tested and its working fine except for one issue. If I try to suggest a chat name it will not rename the chat. Is that set to always use the same model the chat was started with? Maybe it should be set to always use 4o-mini for speed and costs @Niek ?

[Niek]

Thanks for this PR!

We need to check carefully if this doesn't break other models and petals. We can't really hardcode rename to use another model, as that could break other services that don't have it. Why is the summarization breaking?

[Ryan526]

Why is the summarization breaking?

Possibly because maxTokens for suggestName in Chat.svelte is set to 30. Reasoning tokens are probably eating all that up before it can generate a response.

I tried changing that from 30 to 500, won't work... Changed from 500 to 20000 and now it works. Need a way to set reasoning tokens to 0 for chat summarization.

[Ryan526]

https://platform.openai.com/docs/guides/reasoning/managing-the-context-window

If the generated tokens reach the context window limit or the max_completion_tokens value you've set, you'll receive a chat completion response with the finish_reason set to length. This might occur before any visible completion tokens are produced, meaning you could incur costs for input and reasoning tokens without receiving a visible response.

To prevent this, ensure there's sufficient space in the context window or adjust the max_completion_tokens value to a higher number. OpenAI recommends reserving at least 25,000 tokens for reasoning and outputs when you start experimenting with these models. As you become familiar with the number of reasoning tokens your prompts require, you can adjust this buffer accordingly.

Looks like reasoning tokens isn't something that can be controlled. Perhaps under the chat profile could add an option to select a model to use for chat summarization.

[Ryan526]

@Niek it works with all OpenAI models from my testing. I didn't test petals because I don't have and don't use that API.

[Ryan526]

I removed max tokens from chat suggestions so now all models work. Shouldn't be using many tokens anyways on models outside of o1.

[Ryan526]

Can someone that has a petals API verify it still works after these changes? My site is currently on this branch: https://gpt.jalynski.net/

[Niek]

Petals does not require a key, you can simply click the checkbox and use their models. I tried and it seems broken, but TBH so is the main version. I will merge your PR now!

[Niek]

@all-contributors please add @Ryan526 for code

[allcontributors]

@Niek

We had trouble processing your request. Please try again later.

[Niek]

@all-contributors please add @Ryan526 for code

[allcontributors]

@Niek

I've put up a pull request to add @Ryan526! 🎉