feat: Add support for Azure, OpenAI, Palm, Anthropic, Cohere, Replicate Models - using litellm

[ishaan-jaff]

Description

This PR adds support for models from all the above mentioned providers using litellm https://github.com/BerriAI/litellm - a simple & light package to call OpenAI, Azure, Cohere, Anthropic API Endpoints

Here's a sample of how it's used:

from litellm import completion

## set ENV variables
# ENV variables can be set in .env file, too. Example in .env.example
os.environ["OPENAI_API_KEY"] = "openai key"
os.environ["COHERE_API_KEY"] = "cohere key"

messages = [{ "content": "Hello, how are you?","role": "user"}]

# openai call
response = completion(model="gpt-3.5-turbo", messages=messages)

# cohere call
response = completion("command-nightly", messages)

# anthropic call
response = completion(model="claude-instant-1", messages=messages)

Testing

Local testing: Done and passed

Additional context

[ishaan-jaff]

soft-github-policy-service agree company="BerriAI"

[ishaan-jaff]

@deepika087 @dciborow can you please take a look at this PR? Happy to add tests/docs if this initial commit looks good😊

[dciborow]

@deepika087 @dciborow can you please take a look at this PR? Happy to add tests/docs if this initial commit looks good😊

sorry for the delay, will take a look, and will have to review the new package.

I might want to keep the old options around, but we could perhaps add a LITE_LLM env var based toggle for this. Let me ponder.

Update: I'm not sure I see a value in making this change yet. While the package provides ability to call other types of APIs, not sure how the code would handle that. Seems like right now we would still just support Azure and OpenAI.

if you have a way to really clean up this section with your package, while also making it easier to use other services i'll reconsider. For example, would love to see to handle this if/else statement in your code directly instead.

[dciborow]

new package should simplify code further, or provide more features, but only provides parity at cost of increased line count.