Function tool callback #16637
Function tool callback #16637
Conversation
dosubot
bot
added
the
size:L
|
Check out this pull request on See visual diffs & provide feedback on Jupyter Notebooks. Powered by ReviewNB |
| """Async Call.""" | ||
| tool_output = self._fn(*args, **kwargs) | ||
| final_output_content = str(tool_output) | ||
| callback_output = self._run_callback(tool_output) |
There was a problem hiding this comment.
Probably the callback should be async? Otherwise this will block the event loop (probably not ideal)
There was a problem hiding this comment.
In my view, yes. We developed this feature so that when a FunctionTool is called, it can request user input that will influence the result or execution of that function.
That said, it makes sense for it not to be asynchronous. However, in our case, we use synchronous calls. If you believe it should be handled asynchronously, I can change it without any problems.
There was a problem hiding this comment.
I think in a lot of use cases, people are using something like fastapi to serve APIs, and you wouldn't want this callback to halt the entire server.
It probably makes sense to let the user provide either a sync or async callback, and llama-index handles converting it either way (if a sync function is provided, we can make it a "fake" async function with a wrapper. If an async function is provided, we can make it sync using from llama_index.core.utils import asyncio_run and using callback_output = asyncio.run(async_fn(tool_output))
There was a problem hiding this comment.
Stopping to evaluate this point, the person who uses an asynchronous call in this case really doesn't make sense conceptually to stop the application.
So whoever uses the asynchronous callback in the asynchronous call will be in a different situation.
I will make the adjustment
Description
This is a feature that allows applying some human-in-the-loop concepts in FunctionTool.
Basically, a callback function is added that enables the developer to request user input in the middle of an agent interaction, as well as allowing any programmatic action.
New Package?
Did I fill in the
tool.llamahubsection in thepyproject.tomland provide a detailed README.md for my new integration or package?Version Bump?
Did I bump the version in the
pyproject.tomlfile of the package I am updating? (Except for thellama-index-corepackage)Type of Change
Please delete options that are not relevant.
How Has This Been Tested?
Your pull-request will likely not be merged unless it is covered by some form of impactful unit testing.
Suggested Checklist:
make format; make lintto appease the lint gods