Duplicate tool calls, finally hits recursion limit

[umeshshetty]

I have a simple langgprah implementation where I am using the model to bind tools and the expectation is that the model will identify if a tool call is needed and make the call, evaluate the output and respond. The issue I am facing is that the same tools are called repeatedly and finally hits the recursion limit. Strangely it only happens 50% of the time whereas other times it only calls the tool once as expected and returns the output

modelBedrock = ChatBedrockConverse(
    client=bedrock_rt,
    model="us.anthropic.claude-3-5-haiku-20241022-v1:0",
    provider="Anthropic",
    temperature=0,
    max_tokens=None,
).bind_tools(tools=tools, tool_choice=None)


# Define the Template with updated input variables
template = """
TOOLS
------
Assistant can ask the user to use tools to look up information that may be helpful in answering the users original question. The tools the human can use are:

{tools}

You can use a tool for an input only once but you can use the same tool for another input again
If a tool call was requested and an output is received, don't request the same tool again, go to the next tool or provide the final answer
If the question requires using the tools more than once  times please do so by passing multiple tool calls

ACTIONS
------------
The action to take, should be one of [{tool_names}]

FINAL ANSWER
------
If you think you have the answer to the users question, respond with a Final Answer 

USER'S INPUT
--------------------
Here is the user's input (remember to respond with a markdown format)
{messages}
Thought:{agent_scratchpad}
"""

prompt = PromptTemplate(
    template=template,
    input_variables=["messages"],
    partial_variables={
        "agent_scratchpad": [],
        "tools": render_text_description(tools),
        "tool_names": ", ".join([t.name for t in tools]),
    },
)



class AgentState(TypedDict):
    messages: Annotated[List[str], add]  # Only user's questions as strings
    chat_history: Annotated[
        List[Union[HumanMessage, AIMessage, ToolMessage]], add
    ]  # Full chat history
    agent_outcome: Union[AgentAction, str, None]  # Can be AgentAction or description
    intermediate_steps: Annotated[
        List[Tuple[AgentAction, str]], add
    ]  # Intermediate steps


# Create the Langgraph agent executor with the updated prompt
langgraph_AgentAWS = create_react_agent(
    modelBedrock, tools=tools, state_modifier=prompt
)


@observe
def run_agent(state: AgentState) -> dict:
    # Node that calls the LLM

    logging.info("Invoking the LLM with tools \n")
    response = langgraph_AgentAWS.invoke(state)
    logging.info(f"Run Agent Response : {response}")

    logging.info("Appending data to chat_hstory key : \n")
    for msg in response["messages"]:
        state["chat_history"].append(msg)

    logging.info("Updated State : \n")
    return state


workflow = StateGraph(AgentState)
workflow.add_node("run_agent", run_agent)
workflow.set_entry_point("run_agent")
appWorkflow = workflow.compile()


input_question = "What is the weather in Mumbai, California and Frankfurt ?"
agent_input = {
    "messages": [input_question],
    "chat_history": [],
    "agent_outcome": "",
    "intermediate_steps": [],
}
agent_result = appWorkflow.invoke(agent_input, config={"callbacks": [langfuse_handler]})

I can see from langfuse traces the "messages" being appended with AIMessages and ToolMessage outputs but the LLM still decides to make the same tool calls again and again