langgraph/tutorials/multi_agent/hierarchical_agent_teams/

[giscus[bot]]

langgraph/tutorials/multi_agent/hierarchical_agent_teams/

Build language agents as graphs

https://langchain-ai.github.io/langgraph/tutorials/multi_agent/hierarchical_agent_teams/

[gregorybarnes]

The 2nd test to "Write an outline for poem and then write the poem to disk." didn't seem to work with gpt-4o when I initially tried it. I retried using gpt-4 which seemed to work fine. I re-ran the test again using gpt-4o so that I could capture the stack trace and it actually worked the 2nd time around! I'm not 100% certain but I vaguely remember the error complaining about the function call format on the first run.

Anyone else testing with gpt-4o may need to re-run a few times to get it to work correctly.

Also, the last test to "Write a brief research report on the North American sturgeon. Include a chart." triggered rate limit errors when running with gpt-4o:

RateLimitError: Error code: 429 - {'error': {'message': 'Request too large for gpt-4o in organization org-xxxxxxxxxx on tokens per min (TPM): Limit 30000, Requested 38324. The input or output tokens must be reduced in order to run successfully. Visit https://platform.openai.com/account/rate-limits to learn more.', 'type': 'tokens', 'param': None, 'code': 'rate_limit_exceeded'}}

I'm too lazy to add code to correctly handle rate limits...

I wasn't ever able to get the 3rd and final test to complete. gpt-4-1106-preview, gpt-4, and gpt-3.5-turbo all caused the same error on multiple runs:
BadRequestError: Error code: 400 - {'error': {'message': "Invalid 'messages[3].name': string does not match pattern. Expected a string that matches the pattern '^[a-zA-Z0-9_-]+$'.", 'type': 'invalid_request_error', 'param': 'messages[3].name', 'code': 'invalid_value'}}

One time when running the last test with gpt-4-1106-preview I got the following error:
ValueError: An output parsing error occurred. In order to pass this error back to the agent and have it try again, pass handle_parsing_errors=Trueto the AgentExecutor. This is the error: Could not parse tool input: {'arguments': 'scrape_webpages ', 'name': 'functions'} because thearguments is not valid JSON.

I even went back and updated all of the teams to use gpt-4-1106-preview but the 3rd and final test never actually worked.

I get the gist of things here and I understand that I may need to tweak things a bit with prompting but should I be concerned about reliability? So far while running the tutorials in the "Multi-Agent Systems" section things seem pretty brittle and very sensitive/dependent on specific model versions. Are there tricks to getting around this or do I just need to continue working through the other tutorials and learn other design patterns to account for these issues?

[hinthornw]

I don't personally think multi agent architectures are necessary or optimal for most applications, if that's your question

[gregorybarnes]

Ya, I think that cuts down to the core of my question. Basically, what you're suggesting is I should try to define a single agent architecture for most applications because each added layer of agents may expose opportunities and vectors for formatting issues in their responses?

I do have a very specific application that I'm building for Model Evaluations. The goal is to orchestrate multiple agents playing the social deduction game called "Mafia" (sometimes called Werewolf) together.

Do you know if any design patterns tend to be more reliable or optimal compared to others when working with multi agent applications (ex. Supervision design is better than Hierarchical design because it has fewer layers)? Or is reliability more of an exercise in Prompt Engineering (ex. specify example response formats). Or worst case scenario is it an exercise in fine-tuning &/or pre-training?

[hinthornw]

Oh good question. The honest answer is "it depends" and that I'd typically start with fewer moving parts and only add complexity if my evals identify something I can't easily solve with the simpler design.

I think a supervisor pattern or something similar (can do fun things with speaker selection, for instance) would work for this.

1 note: I view prompt engineering and fine-tuning different shades of the same thing -> ways of inducing better task-specific behavior while potentially (increase bias, decrease variance)

[hinthornw]

Cool application btw! Mafia/Werewolf is a really fun game

[goern]

@gregorybarnes could you have a look at #608 and let me know if it fixes your problem?

[HiraveBapu]

@goen

great article. i created simple supervision agent system for arithmetic operation, just to get idea around.

lets say, i have agent "addition", it has its tool to do the job. That tool requires two arguments to run the addition, seems like supervisor and agent stuck in loop during multi-turn conversation

User: add numbers
{'supervisor': {'next': 'Addition'}}
----
{'Addition': {'messages': [HumanMessage(content='Please provide the two numbers you would like to add.', name='Addition')]}}
----
{'supervisor': {'next': 'Addition'}}
----
{'Addition': {'messages': [HumanMessage(content='Please provide the two numbers you would like to add.', name='Addition')]}}
----
{'supervisor': {'next': 'Addition'}}
----

any suggestion ?

[konon4]

In create_agent system_prompt will only include the first line:

    system_prompt += "\nWork autonomously according to your specialty, using the tools available to you."
    " Do not ask for clarification."
    " Your other team members (and other teams) will collaborate with you with their own specialties."
    " You are chosen for a reason! You are one of the following team members: {team_members}."

should be :

    system_prompt += (
        "\nWork autonomously according to your specialty, using the tools available to you."
        " Do not ask for clarification."
        " Your other team members (and other teams) will collaborate with you with their own specialties."
        " You are chosen for a reason! You are one of the following team members: {team_members}."
    )

[terwayp]

def agent_node(state, agent, name):
    result = agent.invoke(state)
    return {"messages": [HumanMessage(content=result["output"], name=name)]}

Why do we need to convert the messages to HumanMessage? Can it be just a BaseMessage?

[HoaNQ98]

I guess that they do that is because the last time, I got an issue with Claude 3.5 Sonnet. He says:

Error raised by bedrock service: An error occurred (ValidationException) when calling the InvokeModel operation: Your API request included an `assistant` message in the final position, which would pre-fill the `assistant` response. When using tools, pre-filling the `assistant` response is not supported.

Each sub supervisor will response a AI message, which is in the final position in messages and call the main supervisor back to act next. But the main supervisor find a last message is AI role, then the model returns the issue above.

I just guess. If I'm right, I also don't like the way: convert AI message to Human message too much.

[terwayp]

class State(TypedDict):
    messages: Annotated[List[BaseMessage], operator.add]
    next: str


def get_last_message(state: State) -> str:
    return state["messages"][-1].content


def join_graph(response: dict):
    return {"messages": [response["messages"][-1]]}


# Define the graph.
super_graph = StateGraph(State)
# First add the nodes, which will do the work
super_graph.add_node("ResearchTeam", get_last_message | research_chain | join_graph)
super_graph.add_node(
    "PaperWritingTeam", get_last_message | authoring_chain | join_graph
)
super_graph.add_node("supervisor", supervisor_node)

What is the logic behind joining graph? Why does return {"messages": [response["messages"][-1]]} help in the join?

[matamorosjuan1]

From what I can tell, the purpose of join graph is to return the last message generated by the chain and add that message to the 'messages' state. Then when the next node is called, the get_last_message function is able to retrieve the content of that last message previously added to the 'messages' state and use that in that processing. Hope that makes sense.

[PragalvhaSharma]

i cannot see the text file it created

[Surizz]

Hi,
I have been following the tutorial on hierarchical agent teams and have a question regarding a specific scenario. When a lower-level agent encounters a human-in-the-loop situation, what is the best approach to handle it?

I am particularly interested in:

1. Ensuring smooth and efficient communication between the agent and the human by checkpoint.

2. Minimizing any delays or interruptions in the overall process.

3. Integrating the human's input back into the system effectively.

Any guidance or examples on this would be greatly appreciated.

[inoue0426]

When I use llm = ChatOllama(model="llama3.1"), I got AttributeError: 'ChatOllama' object has no attribute 'bind_functions'.

When I use llm = ChatOpenAI( api_key="ollama", model="llama3.1", base_url="http://localhost:11434/v1", ) , got this error, ```
File ~/miniconda3/envs/multi/lib/python3.11/site-packages/langchain_core/output_parsers/openai_functions.py:106, in JsonOutputFunctionsParser.parse_result(self, result, partial)
104 return None
105 else:
--> 106 raise OutputParserException(
107 f"Could not parse function call: {exc}"
108 ) from exc
109 try:
110 if partial:

OutputParserException: Could not parse function call: 'function_call'```

Do you know how to deal with this?

[LuisMoralesAlonso]

Hi @inoue0426, got the same issue. My understanding is that bind_functions is deprecated, and with local models you must use bind_tools

This leads to change in the code this way (this works for me):

def create_team_supervisor(llm: ChatOllama, system_prompt, members) -> str:
"""An LLM-based supervisor."""
options = ["FINISH"] + members

# Data model
class RouteSupervisor(BaseModel):
    """Route to the next role one of """ + ', '.join(options)
    
    worker: Literal[tuple(options)] = Field(
        ...,
        description="Next worker, one of " + ', '.join(options),
    )

structured_llm_router = llm.with_structured_output(RouteSupervisor)

# Prompt
prompt = ChatPromptTemplate.from_messages(
    [  
        ("system", system_prompt),
        ("human", "{message}"),
    ]
)
return prompt | structured_llm_router

Once you instantiate your model you can create the supervisor:

#Supervisor Node
research_prompt = "You are a supervisor tasked with managing a conversation between the following workers: Search, WebScraper. Given the following user request, respond with the worker to act next. Each worker will perform a task and respond with their results and status.When finished, respond with FINISH."
supervisor_agent = create_team_supervisor(llm, research_prompt, ["Search", "WebScraper"])

[inoue0426]

Thanks @LuisMoralesAlonso !
I will try this!

[LuisMoralesAlonso]

I'm reviewing the entire use case based on this kind of upgrades... hope i can share more changes in the coming days

[Karlusk]

I got it working but have some questions on why it's working. Specifically, why the supervisor seems stuck in a loop and keeps picking the outline team over and over. It's almost like it cannot see the entire message queue or does not know when to "finish"

It feels like it would be better to have it see the entire message queue and let each helper go once. It just loops.

Am I missing something?

[cris-m]

Try to choose advanced model like gpt-4o or claude-3-5-sonnet as llm for supervisor. I had the same issue but when I change the llm I got the response. I think the solution will be to change the model or change the prompt.

[HoangNguyen689]

Also got stuck because of supervisor in a loop of outline team over and over.
Tested with gpt-4o.
Maybe we need to specify the prompt more precisely.

[iamgauravpant]

Hi guys , I'm trying to add memory to this hierarchical agent system but no success so far .
I tried to integrate the most basic form of memory using in-memory checkpointer but it doesn't work .

Any idea on how to add memory to hierarchical agent system or multi-agent systems in general ?

[Shahzaib-qlu]

let me know if you got any way to do manage it.

[HoangNguyen689]

what do you mean memory? Is that about ask a question that in a thread (has chat history) ?

[Tman910]

Come someone explain the following snippet please? Is there any documentation for how this works?

function_def = {
        "name": "route",
        "description": "Select the next role.",
        "parameters": {
            "title": "routeSchema",
            "type": "object",
            "properties": {
                "next": {
                    "title": "Next",
                    "anyOf": [
                        {"enum": options},
                    ],
                },
            },
            "required": ["next"],
        },
    }

[HamedHaddadi]

This is the json schema of the function passed to the 'bind_function' method.
Consider this simple example:

from pydantic import BaseModel 
from typing import Literal 

class routeResponse(BaseModel):
    next: Literal['one', 'two']

routeResponse.model_json_schema()

{'properties': {'next': {'enum': ['one', 'two'],
   'title': 'Next',
   'type': 'string'}},
 'required': ['next'],
 'title': 'routeResponse',
 'type': 'object'}

[HamedHaddadi]

Just in case someone encounters a same error.
It is not possible to use a lambda as the 'path' to 'add_conditional_edges'.
The best way is to define a function, for instance:
def route_agents(state):
return state['next']
and pass this function as the path to add_conditional_edges.

The reason is in Graph.add_conditional_edges(self, source, path, path_map, then) it requests for name of the RunnableCallable
...
name = path.name or "condition"
and lambda functions are anonymous.

[DanielOsorio01]

Thanks! I had this error too

[Zhuytt20]

why? anyone help me

Traceback (most recent call last):
File "D:\anaconda\envs\lc\Lib\site-packages\langchain_core\output_parsers\openai_functions.py", line 97, in parse_result
function_call = message.additional_kwargs["function_call"]
~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
KeyError: 'function_call'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "D:\langchain_v3.0\43 Hierarchical Agent Teams.py", line 293, in
for s in research_chain.stream(
File "D:\anaconda\envs\lc\Lib\site-packages\langchain_core\runnables\base.py", line 3407, in stream
yield from self.transform(iter([input]), config, **kwargs)
File "D:\anaconda\envs\lc\Lib\site-packages\langchain_core\runnables\base.py", line 3394, in transform
yield from self._transform_stream_with_config(
File "D:\anaconda\envs\lc\Lib\site-packages\langchain_core\runnables\base.py", line 2197, in _transform_stream_with_config
chunk: Output = context.run(next, iterator) # type: ignore
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\anaconda\envs\lc\Lib\site-packages\langchain_core\runnables\base.py", line 3357, in transform
yield from final_pipeline
File "D:\anaconda\envs\lc\Lib\site-packages\langchain_core\runnables\base.py", line 1431, in transform
yield from self.stream(final, config, **kwargs)
File "D:\anaconda\envs\lc\Lib\site-packages\langgraph\pregel_init.py", line 1307, in stream
for _ in runner.tick(
File "D:\anaconda\envs\lc\Lib\site-packages\langgraph\pregel\runner.py", line 56, in tick
run_with_retry(t, retry_policy)
File "D:\anaconda\envs\lc\Lib\site-packages\langgraph\pregel\retry.py", line 29, in run_with_retry
task.proc.invoke(task.input, config)
File "D:\anaconda\envs\lc\Lib\site-packages\langgraph\utils\runnable.py", line 412, in invoke
input = context.run(step.invoke, input, config)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\anaconda\envs\lc\Lib\site-packages\langchain_core\output_parsers\base.py", line 193, in invoke
return self._call_with_config(
^^^^^^^^^^^^^^^^^^^^^^^
File "D:\anaconda\envs\lc\Lib\site-packages\langchain_core\runnables\base.py", line 1927, in _call_with_config
context.run(
File "D:\anaconda\envs\lc\Lib\site-packages\langchain_core\runnables\config.py", line 396, in call_func_with_variable_args
return func(input, **kwargs) # type: ignore[call-arg]
^^^^^^^^^^^^^^^^^^^^^
File "D:\anaconda\envs\lc\Lib\site-packages\langchain_core\output_parsers\base.py", line 194, in
lambda inner_input: self.parse_result(
^^^^^^^^^^^^^^^^^^
File "D:\anaconda\envs\lc\Lib\site-packages\langchain_core\output_parsers\openai_functions.py", line 103, in parse_result
raise OutputParserException(msg) from exc
langchain_core.exceptions.OutputParserException: Could not parse function call: 'function_call'

Process finished with exit code 1

[Silecne666]

I'm currently in the process of hierarchical_agent_teams, which have been successfully completed. However, the llm was found to summarize and optimize the responses returned by the tool. Now I expect the result to be the output returned by the tool without any modifications. Are there any parameters that need to be configured?

[HoaNQ98]

2 ways I think you can try:

1. You write the system prompt for AI that he does not modify any result from his managing tools and keep it as original. Of course, sometimes he can miss that
2. You can consider creating a custom node for your tool that the tool won't need to back to AI. Btw, you probably should configure a conditional edge for that

[Silecne666]

Thanks, I'm going to try both of these methods,
but with debug, there's a ToolMessage, which is what I want,

[bphillab]

After running this I keep getting

{'supervisor': None}

rather than what the expected output is. Does anyone know what's up with that?

[bphillab]

eg for the cat poem:

{'supervisor': None}
---
{'note_taker': {'messages': [HumanMessage(content='The outline for the poem about cats has been created and saved as "cats_poem_outline.txt."', additional_kwargs={}, response_metadata={}, name='note_taker', id='5e6e7f3d-64ad-4216-805b-90b75b23f905')]}}
---
{'supervisor': None}
---
{'doc_writer': {'messages': [HumanMessage(content='The poem about cats has been written and saved as "cats_poem.txt."', additional_kwargs={}, response_metadata={}, name='doc_writer', id='9c7b360e-36ca-47a9-ae13-75714e9db11b')]}}
---
{'supervisor': None}
---

The program is following the graph as expected, but rather than saying the next step for the supervisor it just kinda doesn't.

[HoangHao1009]

Thank you, LangChain/LangGraph team, for the exclusive tutorial! I have a question about the logic behind selecting the next node (i.e., how the supervisor chooses the agent). Is the selection process solely determined by the system prompt, as shown below?

system_prompt = (
"You are a supervisor tasked with managing a conversation between the"
f" following workers: {members}. Given the following user request,"
" respond with the worker to act next. Each worker will perform a"
" task and respond with their results and status. When finished,"
" respond with FINISH."
)

Does the system rely only on the worker names for selection, or are there other factors or mechanisms at play that influence the choice? Additionally, how can I ensure the routing is accurate? Should I make the prompt more concise to improve its effectiveness?

Also, I noticed that the agents (e.g., search_agent and web_scraper_agent) do not seem to have their own system prompts. When is it necessary to add system prompts to agents, and how should I decide whether to include them? I'd appreciate any clarification or tips! 😊

[namkyupar]

Thank you very much for the example.

I tried to change only two things - search engine (duck) and LLM (ollama + llama3.1):

from langchain_community.tools import DuckDuckGoSearchRun
duck_tool = DuckDuckGoSearchRun(max_results=5)
search_agent = create_react_agent(llm, tools=[duck_tool])

from langchain_ollama import ChatOllama
llm = ChatOllama(model="llama3.1", temperature=0)

and I'm getting the following error:

python aaa.py
/Users/namkyup/vsc/langgraph/aaa.py:111: SyntaxWarning: invalid escape sequence '`'
return f"Successfully executed:\n```python\n{code}\n```\nStdout: {result}"
/Users/namkyup/vsc/langgraph/aaa.py:111: SyntaxWarning: invalid escape sequence '`'
return f"Successfully executed:\n```python\n{code}\n```\nStdout: {result}"
USER_AGENT environment variable not set, consider setting it to identify your requests.
<IPython.core.display.Image object>
Traceback (most recent call last):
File "/Users/namkyup/vsc/langgraph/aaa.py", line 202, in
for s in research_graph.stream(
~~~~~~~~~~~~~~~~~~~~~^
{"messages": [("user", "when is Taylor Swift's next tour?")]},
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
{"recursion_limit": 100},
^^^^^^^^^^^^^^^^^^^^^^^^^
):
^
File "/Users/namkyup/vsc/langgraph/lib/python3.13/site-packages/langgraph/pregel/init.py", line 1660, in stream
for _ in runner.tick(
~~~~~~~~~~~^
loop.tasks.values(),
^^^^^^^^^^^^^^^^^^^^
...<2 lines>...
get_waiter=get_waiter,
^^^^^^^^^^^^^^^^^^^^^^
):
^
File "/Users/namkyup/vsc/langgraph/lib/python3.13/site-packages/langgraph/pregel/runner.py", line 167, in tick
run_with_retry(
~~~~~~~~~~~~~~^
t,
^^
...<4 lines>...
},
^^
)
^
File "/Users/namkyup/vsc/langgraph/lib/python3.13/site-packages/langgraph/pregel/retry.py", line 40, in run_with_retry
return task.proc.invoke(task.input, config)
~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^
File "/Users/namkyup/vsc/langgraph/lib/python3.13/site-packages/langgraph/utils/runnable.py", line 408, in invoke
input = step.invoke(input, config, **kwargs)
File "/Users/namkyup/vsc/langgraph/lib/python3.13/site-packages/langgraph/utils/runnable.py", line 184, in invoke
ret = context.run(self.func, input, **kwargs)
File "/Users/namkyup/vsc/langgraph/aaa.py", line 142, in supervisor_node
goto = response["next"]
~~~~~~~~^^^^^^^^
TypeError: 'NoneType' object is not subscriptable
During task with name 'supervisor' and id '661bc734-8df6-309c-b3f3-3316809fa5e5'

I'm also not sure why I get the following two warnings at the beginning:

/Users/namkyup/vsc/langgraph/aaa.py:111: SyntaxWarning: invalid escape sequence '`'
return f"Successfully executed:\n```python\n{code}\n```\nStdout: {result}"
/Users/namkyup/vsc/langgraph/aaa.py:111: SyntaxWarning: invalid escape sequence '`'
return f"Successfully executed:\n```python\n{code}\n```\nStdout: {result}"

and

USER_AGENT environment variable not set, consider setting it to identify your requests.

Any help would be highly appreciated!

[akashAD98]

this code has lots of issue why cant you fix it all ?