[azopenai] LogProbs and TopLogProbs rejected by Azure servers

[davidgwking]

Bug Report

Upon implementing LogProbs and TopLopProbs parameters for Client.ChatCompletions, I am receiving errors from Azure.

• Dependencies

github.com/Azure/azure-sdk-for-go/sdk/ai/azopenai v0.5.0
github.com/Azure/azure-sdk-for-go/sdk/azcore v1.9.2
github.com/Azure/azure-sdk-for-go/sdk/azidentity v1.5.1

• Go Version

❯ go version
go version go1.21.7 darwin/arm64

• What happened?

Upon calling Client.ChatCompletion with azopenai.ChatCompletionsOptions#LogProbs populated, I receive the following error from the SDK:

Type = exported.ResponseError
ErrorCode = ""
StatusCode = 400
JSON Body = {
  "error": {
    "message": "True is not of type 'integer' - 'logprobs'",
    "type": "invalid_request_error",
    "param": null,
    "code": null
  }
}

This is also easily reproducible with the AzureOpenAI Python SDK's ChatCompletions implementation. Due to Python's dynamic typing, I was able to pass an integer to logprobs (to circumvent the boolean issue), but then top_logprobs would be rejected with a different error: Unrecognized argument supplied: top_logprobs. Thus, I believe there are two related issues and it appears they could be bugs in the Azure OpenAI backend, not the SDKs themself.

• What did you expect or want to happen?

I expect that Logprobs is implemented to OpenAI and Azure API spec: it should be a boolean.

I also expect Top Logprobs to be implemented and accepted by the server.

• How can we reproduce it?

azureChatCompletionOptions := &azopenai.ChatCompletionsOptions{
  Messages: []azopenai.ChatRequestMessageClassification{
    &azopenai.ChatRequestUserMessage{
      Content: azopenai.NewChatRequestUserMessageContent(
        "Who are the founders of Microsoft?",
      ),
    },
  },
  DeploymentName: &modelDeploymentName,
  LogProbs:       to.Ptr(true),
  TopLogProbs:    to.Ptr[int32](5),
}

azureResp, err := e.client.GetChatCompletions(azureCtx, *azureChatCompletionOptions, nil)

• Anything we should know about your environment.

I performed tests on these model deployments:

• gpt-35-turbo @ version 0613
• gpt-35-turbo-16k @ version 0613
• gpt-4 @ version 0613

I am utilizing API Version 2024-02-15-preview.

---

Thank you!

[davidgwking]

Hi all. Is there any internal movement/confirmation on this issue?

[richardpark-msft]

Hi @davidgwking, I'll ping back when I get this sorted out, sorry for the long wait on that.

[richardpark-msft]

@davidgwking, just confirmed this now works with the (just released) latest package, which is targeting the latest preview API.

go get github.com/Azure/azure-sdk-for-go/sdk/ai/[email protected]