add with_structured_output support for Pydantic models, dicts and Enum

[mattf]

Bind a structured output schema to the model.

The schema can be -
0. a dictionary representing a JSON schema
1. a Pydantic object
2. an Enum

0. If a dictionary is provided, the model will return a dictionary. Example:

json_schema = {
    "title": "joke",
    "description": "Joke to tell user.",
    "type": "object",
    "properties": {
        "setup": {
            "type": "string",
            "description": "The setup of the joke",
        },
        "punchline": {
            "type": "string",
            "description": "The punchline to the joke",
        },
    },
    "required": ["setup", "punchline"],
}

structured_llm = llm.with_structured_output(json_schema)
structured_llm.invoke("Tell me a joke about NVIDIA")
# Output: {'setup': 'Why did NVIDIA go broke? The hardware ate all the software.',
#          'punchline': 'It took a big bite out of their main board.'}

1. If a Pydantic schema is provided, the model will return a Pydantic object.
   Example:

from langchain_core.pydantic_v1 import BaseModel, Field
class Joke(BaseModel):
    setup: str = Field(description="The setup of the joke")
    punchline: str = Field(description="The punchline to the joke")

structured_llm = llm.with_structured_output(Joke)
structured_llm.invoke("Tell me a joke about NVIDIA")
# Output: Joke(setup='Why did NVIDIA go broke? The hardware ate all the software.',
#              punchline='It took a big bite out of their main board.')

2. If an Enum is provided, all values must be strings, and the model will return
   an Enum object. Example:

import enum
class Choices(enum.Enum):
    A = "A"
    B = "B"
    C = "C"

structured_llm = llm.with_structured_output(Choices)
structured_llm.invoke("What is the first letter in this list? [X, Y, Z, C]")
# Output: <Choices.C: 'C'>

Note about streaming: Unlike other streaming responses, the streamed chunks
will be increasingly complete. They will not be deltas. The last chunk will
contain the complete response.

For instance with a dictionary schema, the chunks will be:

structured_llm = llm.with_structured_output(json_schema)
for chunk in structured_llm.stream("Tell me a joke about NVIDIA"):
    print(chunk)

# Output:
# {}
# {'setup': ''}
# {'setup': 'Why'}
# {'setup': 'Why did'}
# {'setup': 'Why did N'}
# {'setup': 'Why did NVID'}
# ...
# {'setup': 'Why did NVIDIA go broke? The hardware ate all the software.', 'punchline': 'It took a big bite out of their main board'}
# {'setup': 'Why did NVIDIA go broke? The hardware ate all the software.', 'punchline': 'It took a big bite out of their main board.'}

For instnace with a Pydantic schema, the chunks will be:

structured_llm = llm.with_structured_output(Joke)
for chunk in structured_llm.stream("Tell me a joke about NVIDIA"):
    print(chunk)

# Output:
# setup='Why did NVIDIA go broke? The hardware ate all the software.' punchline=''
# setup='Why did NVIDIA go broke? The hardware ate all the software.' punchline='It'
# setup='Why did NVIDIA go broke? The hardware ate all the software.' punchline='It took'
# ...
# setup='Why did NVIDIA go broke? The hardware ate all the software.' punchline='It took a big bite out of their main board'
# setup='Why did NVIDIA go broke? The hardware ate all the software.' punchline='It took a big bite out of their main board.'

For Pydantic schema and Enum, the output will be None if the response is
insufficient to construct the object or otherwise invalid. For instance,

llm = ChatNVIDIA(max_tokens=1)
structured_llm = llm.with_structured_output(Joke)
print(structured_llm.invoke("Tell me a joke about NVIDIA"))

# Output: None

[dglogo]

Do I understand this correctly, and we decided to remove the ability to specify method?
https://python.langchain.com/v0.2/docs/how_to/structured_output/#advanced-specifying-the-method-for-structuring-outputs

[mattf]

The 'method' parameter is unnecessary and is ignored. The appropriate method will be chosen automatically depending on the type of schema provided.

we could enhance this by letting users force a particular implementation, but not all implementations will work with all input types.

[dglogo]

I could be wrong, but specifying JSON mode should force it output JSON regardless. Also this would be consistent with broader lc usage.

[mattf]

the method param effectively requires the user to pick the implementation that is used to achieve their desired outcome of json or pydantic responses from invoke/stream. we can do better than requiring the user to pick.

also, https://python.langchain.com/v0.2/docs/how_to/structured_output/#advanced-specifying-the-method-for-structuring-outputs

"JSON mode" asserts it will not pass the schema to the model, but we will pass the schema to the model.

[dglogo]

ok sgtm!

[dglogo]

Thanks @mattf!