Fix role mapping in GPTAssistantAgent for OpenAI API compatibility

[evandavid1]

Why are these changes needed?

This change addresses a bug where an openai.BadRequestError occurs when using a GPTAssistantAgent and a ConversableAgent that utilizes a tool call within the same GroupChat. Specifically, when the ConversableAgent performs a tool call, messages with internal role 'tool' are added to the chat history. When the GPTAssistantAgent then tries to generate a response, it sends this conversation history to the OpenAI Assistant API, which returns an error because it only accepts messages with roles 'user', 'assistant', per the error message.

Note: Was previously merged to Microsoft autogen repo:
microsoft#3555

Problem Details:
The primary problem occurs when messages with roles like "tool" or "function" are submitted to the OpenAI Assistant API, which expects roles to be either "user" or "assistant". As I was reviewing the best approach to fix this issue, it was clear that modifying the role directly in _append_oai_message can disrupt the internal message history and cause unit tests to fail. Therefore, the recommended solution is to adjust roles within the GPTAssistantAgent when sending to OpenAI API.

Rationale:
By isolating changes to gpt_assistant_agent.py, we avoid impacting other agents and parts of the codebase.
Also we preserve internal logic, so internal AutoGen message roles remain unchanged, ensuring that any internal processing that depends on roles like "function" or "tool" continues to work correctly.

Agents Involved:
A GPTAssistantAgent.
A ConversableAgent that performs tool calls.
A ConversableAgent as user proxy

Error Message:
openai.BadRequestError: Error code: 400 - {'error': {'message': "Invalid value: 'tool'. Supported values are: 'user' and 'assistant'.", 'type': 'invalid_request_error', 'param': 'role', 'code': 'invalid_value'}}

Solution:
Role Mapping: The fix involves mapping internal message roles to roles accepted by the OpenAI Assistant API before sending messages.

Implementation:
Added a _map_role_for_api method in gpt_assistant_agent.py to map internal roles to valid API roles.
Internal roles such as 'tool' and 'function' are mapped to 'assistant'.
Modified the _invoke_assistant method to apply this mapping when constructing messages for the API call.

Benefits:
Ensures compatibility with the OpenAI API by sending only accepted roles, preventing the BadRequestError from occurring. Allows the GPTAssistantAgent to function correctly within a groupchat that includes ConversableAgents utilizing tool calls.

Software versions:
pyautogen=0.2.35
python=3.12.2
tested with both openai=1.30.1 and openai=1.47.0
tested with gpt-4o and gpt-4o-mini

Logging:
if you add logging to the mre program, you can see the role issue in the log:
logging:
import logging
logging.basicConfig(level=logging.DEBUG)

output before fix: (role='tool')
DEBUG:openai._base_client:Request options: {'method': 'post', 'url': '/threads/thread_hSK0dCkoMCgRTwXDjxQOeV8v/messages', 'headers': {'OpenAI-Beta': 'assistants=v2'}, 'files': None, 'json_data': {'content': "This is a message from the 'print_message' function.", 'role': 'tool'}}

after fix: (role='system')
'role': 'user', 'name': 'Analyst'}, {'content': "Read the above conversation. Then select the next role from ['user_proxy', 'FunctionAgent', 'Analyst'] to play. Only return the role.", 'name': 'checking_agent', 'role': 'system'}], 'model': 'gpt-4o', 'stream': False, 'temperature': 0.0}}

A Minimal Reproducible Example (MRE) was created to demonstrate the issue and verify that the fix works consistently.

MRE Overview:
Creates an openai assistant analyst agent, then plugs in the asssistant id into the gpt assistant in the groupchat.
Set up GroupChat with GPTAssistantAgent and a ConversableAgent that performs a tool call.
The ConversableAgent invokes a tool, adding a message with role 'tool' to the chat history.
When the GPTAssistantAgent attempts to process the chat history, it sends the messages to the OpenAI API.
Without the fix, the API returns a BadRequestError due to unrecognized roles.
With the fix, the internal roles are correctly mapped, and no error occurs.

MRE:

from openai import OpenAI
from autogen import ConversableAgent, GroupChat, GroupChatManager, config_list_from_json
from autogen.agentchat.contrib.gpt_assistant_agent import GPTAssistantAgent

# OAI_CONFIG_LIST Example:
# Ensure that your OAI_CONFIG_LIST is properly formatted as shown below.
#
# [
#     {
#         "model": "gpt-4o",
#         "api_key": "INSERT YOUR OPENAI KEY"
#     }
# ]
# ------------------------------------------------------------------------

# Load LLM configuration from environment or file
config_list = config_list_from_json(env_or_file="OAI_CONFIG_LIST")
llm_config = {
    "config_list": config_list,
    "temperature": 0.0,
}

# Extract the assistant-specific configuration
if not config_list:
    raise ValueError("No configuration found in OAI_CONFIG_LIST.")

assistant_config = config_list[0]

api_key = assistant_config.get("api_key")
model = assistant_config.get("model")

client = OpenAI(api_key=api_key)

# Step 1: Create a new Assistant
assistant = client.beta.assistants.create(
    name="Analyst",
    instructions="""You are an insightful analyst. Your goal is to offer clear, concise, and accurate
    explanations based on the data you gather, ensuring that responses are well-structured and directly address the user's request.""",
    model=model,
)

# Get the Assistant ID
assistant_id = assistant.id
print(f"Assistant ID: {assistant_id}")

# Define the User Proxy Agent
user_proxy = ConversableAgent(
    name="user_proxy",
    description="Represents the user and manages interactions between the user and other agents.",
    llm_config=llm_config,
    human_input_mode="ALWAYS",
)

# Agent that can call a function
function_agent = ConversableAgent(
    name="FunctionAgent",
    description="An agent that can call a function called 'print_message'.",
    system_message="You are an agent that uses the 'print_message' function to provide information.",
    llm_config=llm_config,
    human_input_mode="NEVER",
)

# Register the function
@user_proxy.register_for_execution()
@function_agent.register_for_llm(
    name="print_message", description="Prints a predefined message."
)
def print_message() -> str:
    """A sample function that returns a predefined message."""
    return "This is a message from the 'print_message' function."

# Analyst Agent
analyst = GPTAssistantAgent(
    name="Analyst",
    description="Provides additional analysis after the function is called.",
    llm_config={
        "config_list": llm_config["config_list"],
        "assistant_id": assistant_id,
    },
)

# Set up the group chat with the agents
groupchat = GroupChat(
    agents=[user_proxy, function_agent, analyst],
    messages=[],
    max_round=10,
)

def main():
    # Task to initiate the chat and reproduce the bug
    task = "Please execute the function and then provide additional analysis."

    manager = GroupChatManager(
        groupchat=groupchat, llm_config=llm_config
    )

    user_proxy.initiate_chat(
        manager,
        message=task,
    )

if __name__ == "__main__":
    main()

Testing:
All unit tests have been run locally to ensure existing functionality remains unaffected.

Related issue number

Closes microsoft#3284
microsoft#3284

Checks

• No doc changes
• No unit tests modified
• I've run precommit, made sure all auto checks have passed, that no unit tests have broken as a result of the fix and code coverage does not decrease.

[sonichi]

@evandavid1 is it good to have a test that covers the new case supported in the contrib-openai CI?

[evandavid1]

@evandavid1 is it good to have a test that covers the new case supported in the contrib-openai CI?

Hi @sonichi yes, good idea. Will add tests and include going forward

[qingyun-wu]

@evandavid1, thanks for the great contribution! Are you working on adding tests? Also, I see that there is a conflict in the file autogen/agentchat/contrib/gpt_assistant_agent.py. Could you fix the conflict? Thank you!

[evandavid1]

Hi - sorry for delay, reviewing conflict and will submit tests

…

On Sun, Oct 20, 2024 at 11:43 AM Qingyun Wu ***@***.***> wrote: @evandavid1 <https://github.com/evandavid1>, thanks for the great contribution! Are you working on adding tests? Also, I see that there is a conflict in the file autogen/agentchat/contrib/gpt_assistant_agent.py. Could you fix the conflict? Thank you! — Reply to this email directly, view it on GitHub <#46 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ANXHUETSWT24E4PCJ27YJLDZ4P2T5AVCNFSM6AAAAABPCCFYXCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDIMRVGE3TIMJXGE> . You are receiving this because you were mentioned.Message ID: ***@***.***>