Using CheckPointer makes the tool call break. #544
Replies: 3 comments 22 replies
-
|
Thanks for raising! Tool handling is a tricky aspect of how updates and resumptions interact with the LLMs' tool calling APIs that I should probably explicitly address in the docs - once an ai message with tool calls is put in the state, the next time the LLM is called, a tool message with the corresponding id must be present in the chat history . If the tool is actually called, then that handles it. If we skip (i.e., with interrupts), then we need to add a tool message to satisfy this API requirement (can say "OK" or anything else) Without seeing your code, if I had to guess, I would assume perhaps there is an interrupt but then when you resume you are inserting a human message or skipping the tool message entirely. Action item for me: make How-To on managing state updates with tool calling I think this is what is causing your issue (based on personal experience) but feel free to follow up if there's actually a break that you identify |
Beta Was this translation helpful? Give feedback.
-
|
Hi @hinthornw ! Indeed, after reading your reply, I checked and the issue occurs if there is an interrupted tool call. Please let me know if I can do anything to help resolve the issue. |
Beta Was this translation helpful? Give feedback.
-
|
Gonna convert this to a discussion for now since it seems to not be a bug in the API so much as a tricky implementation detail we ought to improve in our documentation/guides |
Beta Was this translation helpful? Give feedback.
-
|
I hit this issue as well while adding logic to "prune" an agent's historical message context with a Postgres checkpointer. I got around it for now with a somewhat naive method to ensure that the pruning logic also removes the tool message when it would otherwise be removing only an def prune_historical_messages(
messages: List[AnyMessage],
num_most_recent_messages: int,
) -> List[AnyMessage]:
"""
Prune historical messages while maintaining the integrity of tool-calling conversations.
This function removes older messages, keeping only the specified number of most recent messages.
It ensures that the pruning process doesn't break the logical flow of the conversation,
particularly in the context of tool-calling interactions.
Args:
messages (List[AnyMessage]): The list of messages to be pruned.
num_most_recent_messages (int): The number of most recent messages to keep.
Returns:
List[AnyMessage]: A pruned list of messages.
Note:
- The function preserves the pairing of AI messages with tool calls and their corresponding tool messages.
- If the total number of messages is less than or equal to num_most_recent_messages, no pruning occurs.
- The function adjusts the starting index to avoid leaving a "dangling tool message" at the beginning of the pruned list.
"""
# Check boundary conditions for quick return.
if not len(messages) or len(messages) <= num_most_recent_messages:
return messages
# Ensure that we do not leave a disjoint history state of a "dangling tool message".
idx_start = max(0, len(messages) - num_most_recent_messages)
while idx_start < len(messages) and messages[idx_start].type == "tool":
idx_start += 1
return messages[idx_start:] |
Beta Was this translation helpful? Give feedback.
-
|
Well, I've come to this conclusion, langchain/langgraph are nowhere near production-ready tools, and should be avoided whenever possible. |
Beta Was this translation helpful? Give feedback.
-
|
@ozankasikci the issue is not with langgraph. langgraph is very low level. by itself it does not handle clean up of the message state, that is not an expectation of langgraph things like create_react_agent are higher level abstractions, and cleaning up message state should live there. we are working on adding it into there, but if you want to add your particular logic for handling message state it is pretty easy to do on the base, core langgraph lmk if that distinction doesnt make sense |
Beta Was this translation helpful? Give feedback.
-
|
@hwchase17 hi, it's been 5 months since I reported this issue, and no solution is on the horizon yet. This is a pretty long time, for an issue as obvious as this one. Moreover, all the abstraction implemented in Langchain becomes a burden as the project needs to grow. Just studying all these layers of abstraction and going through the awful documentation of Langchain almost becomes a full-time job. I gave up on Langchain and decided to go with plain OpenAI API, which is straightforward, and clear. Also, the codebase is much more maintainable and extensible as opposed to the locked-in abstractions with limitations and bugs in Langchain. Don't get me wrong, not a hater here. Langchain is still good for some quick prototyping, but not for a production application, I think. |
Beta Was this translation helpful? Give feedback.
-
|
Hi @ozankasikci , hitting this issue and it has been a slippery slope trying to grasp all the abstractions in langgraph. how did you handle things like memory and interruption with just Open AI 's APIs? |
Beta Was this translation helpful? Give feedback.
-
Checked other resources
Example Code
Error Message and Stack Trace (if applicable)
Description
I'm trying to use langchain to build an assistant AI agent.
If I don't use checkpointer, tool calls work just fine.
If I set a SqliteSaver as the checkpointer, the tool calls fail.
System Info
Beta Was this translation helpful? Give feedback.
All reactions