Structured message type in AgentChat

[ekzhu]

What feature would you like to be added?

AgentChat agents can emit and return a message that contains an object rather than a text field.

For example:

ContentTypeT = TypeVar("ContentTypeT", bound=BaseModel)

class StructuredMessage(BaseChatMessage, Generic[ContentTypeT]):
    content: ContentTypeT
    source: str

So, we can instantiate a message like StructuredMessage[Leger](content=leger, source="magentic-one-orchestrator") for magentic one orchestrator to emit to the stream. #5127

We can also make it possible for an agent to return a structured tool call input as a structured message.

Why is this needed?

Many applications require agent to communicate via structured messages, and agents may emit structured "inner thoughts" for better observability.

Right now, the only way to emit "structured message" is through a serialized JSON in a TextMessage field. A structured message type helps the consumer understand what to expect in the message itself. This also allows for filtering at the consumer of the stream to decide whether to display the message, e.g., to the console.

[victordibia]

Is this related to #4725 ?
That PR is around having a structured response in ToolCallExecution Result ?
This way tools can return attachments e.g, pdf files, etc

[ekzhu]

That PR is around having a structured response in ToolCallExecution Result ?

I think a structured response would be able to handle the followings:

• Tool call request: sometime it is good to use the tool call feature to force the model to generate a structured response that can be used to call a tool outside of the agent. E.g., a scenario when the agent is on the server and the tool is at the client side.
• Tool call result: a tool result be more than just a string, as you pointed out.
• General response: many custom agents may perform special operations such as retrieval, web searches, etc., that returns a structured result.