Azure OpenAI as llm

[Jamewsss]

Hi, the configuration page lists azure as a supported llm but there isn't any examples of how to configure azure in the config.yml file, I've been attempting but keep getting a resource not found error. Has anyone managed to implement Azure with NeMo?

[trebedea]

Hi @Jamewsss , here is an example on using Azure OpenAI:
#72

However a small number of rails, e.g. SelfCheck hallucination rail, only work with OpenAI models from langchain. As far as you don't have to use these rails, this solution should work. Can you check it and let us know?

[Jamewsss]

Hi @trebedea thank you for sharing this, this method worked for me using the Output rails.

[trebedea]

This MR is to be merged today which makes using Azure OpenAI even easier: #242
Closing discussion for now.

[onkarrajambatwar]

When this change will be available?

I am pointing to dev branch and trying to run nemoguardrails chat --config=config/ however, I am getting error from

│ ❱ 146 │ │ │ │ self.llm = provider_cls(**kwargs)

And the error is ValidationError: 1 validation error for AzureOpenAI
root
Did not find openai_api_version, please add an environment variable OPENAI_API_VERSION which contains it, or
pass openai_api_version as a named parameter. (type=value_error)

Could you please help.

I have added yml as below
models:

• type: main
  engine: azure
  model: model_name
  parameters:
  azure_endpoint:
  api_version:
  deployment_name: model_name
  api_key:

[trebedea]

Looks like you need to specify the OpenAI API version either as a parameter in the yml configuration or as OPENAI_API_VERSION environment variable. Details on API versions in the Azure documentation:

https://learn.microsoft.com/en-US/azure/ai-services/openai/reference

If you add it in the yml file it should look something like below (in your example the api_version param is empty).

type: main
engine: azure
model: gpt-4-1106-preview
parameters:
    azure_endpoint:
    api_version: 2023-07-01-preview
    deployment_name:
    api_key: <YOUR_API_KEY>