Compatibility with "tools" (function calls)

[gaborvar]

Hi, I am trying to add to this great sample app the capability to call a function, as documented in https://cookbook.openai.com/examples/how_to_call_functions_with_chat_models

I added code:

1. to add the function definition to model_args.
2. to format_non_streaming_response() (and soon to format_stream_response() ) in utils.by to detect when the model wants to call the function.
3. In app.by in send_chat_request() messages with "tool" role are dropped before the call is made to the model. I removed this filtering.

The core functionality of calling a custom code works fine. However I need add information about the function call to the message log so the model is aware later.
The sample app only logs completions by "assistant" into the list of messages, so I will add code to allow completions with "tool" role to be added. I am hoping that completions with "context" field, which are currently logged with "tool" role will not be affected.

Questions:

1. The message log is passed through the frontend. If the frontend makes assumptions on message list format (response_obj) I may break it. I do not have good visibility into the frontend.
2. Streaming: Is this approach compatible with streaming?
3. Is this the right approach? Anything I am missing? Does this sample app have the flexibility to accommodate changes needed to add function calling ?
4. Anybody working on a similar effort?

Thank you!