Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for max_completion_tokens in Azure OpenAI #6376

Merged
merged 1 commit into from
Oct 22, 2024

Conversation

taralika
Copy link
Contributor

Now that Azure supports max_completion_tokens, no need for special handling for this param and let it pass thru. More details: https://learn.microsoft.com/en-us/azure/ai-services/openai/concepts/models?tabs=python-secure#api-support

Relevant issues

Fixes #6374

Type

🆕 New Feature
🐛 Bug Fix

Changes

[REQUIRED] Testing - Attach a screenshot of any new tests passing locall

If UI changes, send a screenshot/GIF of working UI fixes

Now that Azure supports `max_completion_tokens`, no need for special handling for this param and let it pass thru. More details: https://learn.microsoft.com/en-us/azure/ai-services/openai/concepts/models?tabs=python-secure#api-support
Copy link

vercel bot commented Oct 22, 2024

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
litellm ✅ Ready (Inspect) Visit Preview 💬 Add feedback Oct 22, 2024 3:30pm

@krrishdholakia krrishdholakia changed the base branch from main to litellm_dev_10_22_2024 October 22, 2024 19:11
@krrishdholakia krrishdholakia merged commit 9a9caf9 into BerriAI:litellm_dev_10_22_2024 Oct 22, 2024
2 of 3 checks passed
krrishdholakia added a commit that referenced this pull request Oct 23, 2024
* fix(utils.py): add 'disallowed_special' for token counting on .encode()

Fixes error when '<
endoftext
>' in string

* Revert "(fix) standard logging metadata + add unit testing  (#6366)" (#6381)

This reverts commit 8359cb6.

* add new 35 mode lcard (#6378)

* Add claude 3 5 sonnet 20241022 models for all provides (#6380)

* Add Claude 3.5 v2 on Amazon Bedrock and Vertex AI.

* added anthropic/claude-3-5-sonnet-20241022

* add new 35 mode lcard

---------

Co-authored-by: Paul Gauthier <[email protected]>
Co-authored-by: lowjiansheng <[email protected]>

* test(skip-flaky-google-context-caching-test): google is not reliable. their sample code is also not working

* Fix metadata being overwritten in speech() (#6295)

* fix: adding missing redis cluster kwargs (#6318)

Co-authored-by: Ali Arian <[email protected]>

* Add support for `max_completion_tokens` in Azure OpenAI (#6376)

Now that Azure supports `max_completion_tokens`, no need for special handling for this param and let it pass thru. More details: https://learn.microsoft.com/en-us/azure/ai-services/openai/concepts/models?tabs=python-secure#api-support

* build(model_prices_and_context_window.json): add voyage-finance-2 pricing

Closes #6371

* build(model_prices_and_context_window.json): fix llama3.1 pricing model name on map

Closes #6310

* feat(realtime_streaming.py): just log specific events

Closes #6267

* fix(utils.py): more robust checking if unmapped vertex anthropic model belongs to that family of models

Fixes #6383

* Fix Ollama stream handling for tool calls with None content (#6155)

* test(test_max_completions): update test now that azure supports 'max_completion_tokens'

* fix(handler.py): fix linting error

---------

Co-authored-by: Ishaan Jaff <[email protected]>
Co-authored-by: Low Jian Sheng <[email protected]>
Co-authored-by: David Manouchehri <[email protected]>
Co-authored-by: Paul Gauthier <[email protected]>
Co-authored-by: John HU <[email protected]>
Co-authored-by: Ali Arian <[email protected]>
Co-authored-by: Ali Arian <[email protected]>
Co-authored-by: Anand Taralika <[email protected]>
Co-authored-by: Nolan Tremelling <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Feature]: Support Azure OpenAI max_completion_tokens param
2 participants