docs: Add example using TypedDict in structured outputs how-to guide

[barseghyanartur]

For me, the Pydantic example does not work (tested on various Python versions from 3.10 to 3.12, and Pydantic versions from 2.7 to 2.9).

The TypedDict example (added in this PR) does.

---

Additionally, fixed an error in Using PydanticOutputParser example.

Was:

query = "Anna is 23 years old and she is 6 feet tall"

print(prompt.invoke(query).to_string())

Corrected to:

query = "Anna is 23 years old and she is 6 feet tall"

print(prompt.invoke({"query": query}).to_string())

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
langchain	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Nov 13, 2024 4:53pm

[barseghyanartur]

@baskaryan

Please, review this PR.

[eyurtsev]

@barseghyanartur what do you mean by it not working?

[barseghyanartur]

It fails with an error. Would you like to see the traceback?

[barseghyanartur]

@eyurtsev:

It fails with an error. Would you like to see the traceback?

[eyurtsev]

Yes if you have it handy

[barseghyanartur]

@eyurtsev

Two identical examples. One with OpenAI, another one with Ollama.

---

This works

Filename: simple_pipeline_structured_output/pipeline_openai_joke_multiple_schemas_pydantic.py

from pprint import pprint
from typing import Any, Union, Optional

from pydantic import BaseModel, Field
from langchain_openai import ChatOpenAI

__all__ = ("main",)

model = ChatOpenAI(model="gpt-4o-mini")


class Joke(BaseModel):
    """Joke to tell user."""

    setup: str = Field(description="The setup of the joke")
    punchline: str = Field(description="The punchline to the joke")
    rating: Optional[int] = Field(
        default=None, description="How funny the joke is, from 1 to 10"
    )


class ConversationalResponse(BaseModel):
    """Respond in a conversational manner. Be kind and helpful."""

    response: str = Field(description="A conversational response to the user's query")


class FinalResponse(BaseModel):
    final_output: Union[Joke, ConversationalResponse]


structured_llm = model.with_structured_output(FinalResponse)


def main() -> dict[str, Any]:
    """Entrypoint of the pipeline."""
    joke_result = structured_llm.invoke("Tell me a joke about cats")

    print("\n" + "*" * 26 + "\n")
    print("joke_result:")
    pprint(joke_result)

    conversational_result = structured_llm.invoke("How are you today?")

    print("\n" + "*" * 26 + "\n")
    print("conversational_result:")
    pprint(conversational_result)
    return {
        "joke_result": joke_result,
        "conversational_result": conversational_result,
    }


if __name__ == "__main__":
    main()

---

This fails

Filename: simple_pipeline_structured_output/pipeline_ollama_joke_multiple_schemas_pydantic.py

from pprint import pprint
from typing import Any, Union, Optional

from pydantic import BaseModel, Field
from langchain_ollama import ChatOllama

__all__ = ("main",)

model = ChatOllama(model="llama3.1")


class Joke(BaseModel):
    """Joke to tell user."""

    setup: str = Field(description="The setup of the joke")
    punchline: str = Field(description="The punchline to the joke")
    rating: Optional[int] = Field(
        default=None, description="How funny the joke is, from 1 to 10"
    )


class ConversationalResponse(BaseModel):
    """Respond in a conversational manner. Be kind and helpful."""

    response: str = Field(description="A conversational response to the user's query")


class FinalResponse(BaseModel):
    final_output: Union[Joke, ConversationalResponse]


structured_llm = model.with_structured_output(FinalResponse)


def main() -> dict[str, Any]:
    """Entrypoint of the pipeline."""
    joke_result = structured_llm.invoke("Tell me a joke about cats")

    print("\n" + "*" * 26 + "\n")
    print("joke_result:")
    pprint(joke_result)

    conversational_result = structured_llm.invoke("How are you today?")

    print("\n" + "*" * 26 + "\n")
    print("conversational_result:")
    pprint(conversational_result)
    return {
        "joke_result": joke_result,
        "conversational_result": conversational_result,
    }


if __name__ == "__main__":
    main()

---

Trace:

Traceback (most recent call last):
  File "/Users/me/repos/langchain_tryouts/simple_pipeline_structured_output/pipeline_ollama_joke_multiple_schemas_pydantic.py", line 55, in <module>
    main()
  File "/Users/me/repos/langchain_tryouts/simple_pipeline_structured_output/pipeline_ollama_joke_multiple_schemas_pydantic.py", line 37, in main
    joke_result = structured_llm.invoke("Tell me a joke about cats")
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/me/.virtualenvs/langchain-tryouts/lib/python3.11/site-packages/langchain_core/runnables/base.py", line 3024, in invoke
    input = context.run(step.invoke, input, config)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/me/.virtualenvs/langchain-tryouts/lib/python3.11/site-packages/langchain_core/output_parsers/base.py", line 193, in invoke
    return self._call_with_config(
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/me/.virtualenvs/langchain-tryouts/lib/python3.11/site-packages/langchain_core/runnables/base.py", line 1927, in _call_with_config
    context.run(
  File "/Users/me/.virtualenvs/langchain-tryouts/lib/python3.11/site-packages/langchain_core/runnables/config.py", line 396, in call_func_with_variable_args
    return func(input, **kwargs)  # type: ignore[call-arg]
           ^^^^^^^^^^^^^^^^^^^^^
  File "/Users/me/.virtualenvs/langchain-tryouts/lib/python3.11/site-packages/langchain_core/output_parsers/base.py", line 194, in <lambda>
    lambda inner_input: self.parse_result(
                        ^^^^^^^^^^^^^^^^^^
  File "/Users/me/.virtualenvs/langchain-tryouts/lib/python3.11/site-packages/langchain_core/output_parsers/openai_tools.py", line 298, in parse_result
    raise e
  File "/Users/me/.virtualenvs/langchain-tryouts/lib/python3.11/site-packages/langchain_core/output_parsers/openai_tools.py", line 293, in parse_result
    pydantic_objects.append(name_dict[res["type"]](**res["args"]))
                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/me/.virtualenvs/langchain-tryouts/lib/python3.11/site-packages/pydantic/main.py", line 212, in __init__
    validated_self = self.__pydantic_validator__.validate_python(data, self_instance=self)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
pydantic_core._pydantic_core.ValidationError: 2 validation errors for FinalResponse
final_output.Joke
  Input should be a valid dictionary or instance of Joke [type=model_type, input_value='Why did the cat join a b...be the purr-cussionist!', input_type=str]
    For further information visit https://errors.pydantic.dev/2.9/v/model_type
final_output.ConversationalResponse
  Input should be a valid dictionary or instance of ConversationalResponse [type=model_type, input_value='Why did the cat join a b...be the purr-cussionist!', input_type=str]
    For further information visit https://errors.pydantic.dev/2.9/v/model_type

[barseghyanartur]

@eyurtsev

P.S.

Same with TypedDict works without errors for both OpenAI and Ollama.

[eyurtsev]

confirmed and created an issue here: #28090

[eyurtsev]

Updated language in the how-to guide

[eyurtsev]

@barseghyanartur this is likely llama-3.1 just not being good enough and not following the structured output schema. TODO item is to figure out how to improve the error messages so it's easier for users to understand that they're bumping against a model limitation.