OpenAI AzureChatOpenAI doesn't support the new structured output capability even though BaseChatOpenAI does

[mgberg]

Checked other resources

• I added a very descriptive title to this issue.
• I searched the LangChain documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.
• I am sure that this is a bug in LangChain rather than my code.
• The bug is not resolved by updating to the latest stable version of LangChain (or the specific integration package).

Example Code

The following code doesn't work:

class Person(BaseModel):
    first_name: str
    last_name: str


LLM = AzureChatOpenAI(
    azure_deployment="gpt-4o-2024-08-06", api_version="2024-08-01-preview", temperature=0
).with_structured_output(schema=Person, method="json_schema", strict=True)
result = LLM.invoke(HumanMessage("Who was the first president of the United States?"))

Error Message and Stack Trace (if applicable)

The exception received was:

Received unsupported arguments {'strict': True}
  File "/home/goldbermg3/GitProjects/remis-pdf-utils/structured_out_test.py", line 13, in <module>
    ).with_structured_output(schema=Person, method="json_schema", strict=True)
      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ValueError: Received unsupported arguments {'strict': True}

Description

I expected the new Structured Output capability (which is available through Azure) which is implemented on BaseChatOpenAI to work using AzureChatOpenAI. However, after browsing the code, I saw that the support for mode="json_schema" and the strict argument is implemented only for BaseChatOpenAI; AzureChatOpenAI overrides BaseChatOpenAI.with_structured_outputs and does not include support for those arguments.

System Info

System Information

OS: Linux
OS Version: #1 SMP Fri Mar 29 23:14:13 UTC 2024
Python Version: 3.11.9 (main, Apr 6 2024, 17:59:24) [GCC 11.4.0]

Package Information

langchain_core: 0.2.35
langchain: 0.2.15
langsmith: 0.1.104
langchain_openai: 0.1.23
langchain_text_splitters: 0.2.2

Optional packages not installed

langgraph
langserve

Other Dependencies

aiohttp: 3.10.5
async-timeout: Installed. No version info available.
httpx: 0.27.0
jsonpatch: 1.33
numpy: 1.26.4
openai: 1.42.0
orjson: 3.10.7
packaging: 24.1
pydantic: 2.8.2
PyYAML: 6.0.2
requests: 2.32.3
SQLAlchemy: 2.0.32
tenacity: 8.5.0
tiktoken: 0.7.0
typing-extensions: 4.12.2

[arne-rustad]

Is there any plan for this to be fixed?

[PvanHengel]

Whats the update here?

[arne-rustad]

How can we get this fixed? I would assume most businesses in Europe do not use the OpenAI API directly, but instead through Azure. Thus, this is a pretty bad limitation that it is not implemented for the Azure version of the API.

[Emilyho11]

Hello, my group and I are from the University of Toronto, and we are going to take a look and work on this issue, hopefully to have a working PR soon.

[arne-rustad]

Hello, my group and I are from the University of Toronto, and we are going to take a look and work on this issue, hopefully to have a working PR soon.

Hi Emily, I really appreciate you volunteering to work on this. However, I believe the issue has been fixed already. I've seen no update about it, but I was checking the code and saw that it was updated. Tested it a couple of days ago and it appears to now be working. It only works for GPT-4o, not GPT-4o-mini, but that is an Azure issue, not a Langchain issue. Even though OpenAI released strict structured output for GPT-4o-mini two months ago, Azure is yet to do the same.

[baskaryan]

should be fixed as of #26599 which was released in langchain-openai 0.2.2. @arne-rustad @PvanHengel @mgberg could you check if that fixes?

[muriloime]

@baskaryan I confirm this has worked ( had to bump the api version to 2024-08-01-preview )