Sending a (seemingly) valid toolChoice object produces an API error

[jmaczan]

• Package Name: @azure/openai
• Package Version: tested on 1.0.0-beta.8 and 1.0.0-beta.10
• Operating system: Windows 10
• nodejs
  • version: 18.12.1
• [] browser
  • name/version:
• typescript
  • version: 4.9.5
• Is the bug related to documentation in
  • README.md
  • source code documentation
  • SDK API docs on https://docs.microsoft.com

Describe the bug
When sending toolChoice in third parameter object of getChatCompletions, I get an API error:

{
    message: "'$.tool_choice' is invalid. Please check the API 
reference: https://platform.openai.com/docs/api-reference.",   
    type: 'invalid_request_error',
    param: null,
    code: null
  }

toolChoice is defined as optional parameter in GetChatCompletionsOptions and it might look like this:

export declare interface ChatCompletionsNamedFunctionToolSelection {
    /** The object type, which is always 'function'. */
    type: "function";
    /** The name of the function that should be called. */
    name: string;
}

The value is send is

toolChoice: {
    type: "function",
    name: "someFunctionName",
}

To Reproduce
Steps to reproduce the behavior:

1. Create a simple NodeJS app that sends requests to Azure OpenAI API:

import {
    AzureKeyCredential,
    ChatRequestMessage,
    OpenAIClient,
} from "@azure/openai";


try {
    const client = new OpenAIClient(
        process.env.AOAIEndpoint, // fill with your value
        new AzureKeyCredential(process.env.AOAIKey) // fill with your value
    );

    const deploymentId = process.env.AOAIDeploymentId; // fill with your value

      let result = await client
          .getChatCompletions(
              deploymentId,
              [{ role: "user", content: "Call my function", }] as ChatRequestMessage[],
              {
                  toolChoice: {
                      type: "function",
                      name: "someFunctionName",
                  },
                  tools: [
                      {
                          function: someFunction,  // 
                          type: "function",
                      },
                  ],
              }
          )

And you need to specify someFunction:

import { FunctionDefinition } from "@azure/openai";

export const someFunction: FunctionDefinition = {
    name: "someFunctionName",
    description:
        "This is a pretty nice function to call",
    parameters: {
        type: "object",
        properties: {
            question: {
                type: "string",
                description:
                    "A question that user asked to you",
            },
        },
    },
};

Expected behavior
The worst scenario is that toolChoice parameter is ignored by API. The best is that it works just like in vanilla OpenAI API - https://platform.openai.com/docs/api-reference/chat/create#chat-create-tool_choice, meaning it allows to define when tool should be called

Screenshots

Additional context

Tasks

Preview Give feedback

No tasks being tracked yet.

[deyaaeldeen]

@jmaczan thanks for opening this report! Could you please share the exact model your deployment utilizes?

[jmaczan]

@deyaaeldeen Much appreciate your quick response! I'm using GPT-4 1106-Preview. And I'm currently working on a PR fixing this btw, but I'm new to your tooling (like rush) and so my project is still building yet. Do you accept PRs from the outside of MSFT to this repo?

[xirzec]

Do you accept PRs from the outside of MSFT to this repo?

Orthogonal to the issue in this thread, I wanted to answer this question specifically to say yes, we do!

[deyaaeldeen]

@jmaczan @azure/[email protected] has been released with the fix. Please give it a try and let us know if you have any other questions or concerns!

[github-actions]

Hi @jmaczan. Thank you for opening this issue and giving us the opportunity to assist. We believe that this has been addressed. If you feel that further discussion is needed, please add a comment with the text "/unresolve" to remove the "issue-addressed" label and continue the conversation.

[jmaczan]

Much thanks for cooperation on this issue, I’m happy I could help. All the best to you and the team!