Autogen Framework Guide
Autogen Framework Guide
In this and the following section, we focus on the core concepts of AutoGen: agents, agent
runtime, messages, and communication – the foundational building blocks for an multi-agent
applications.
Note
The Core API is designed to be unopinionated and flexible. So at times, you may find it
challenging. Continue if you are building an interactive, scalable and distributed multi-agent
system and want full control of all workflows. If you just want to get something running
quickly, you may take a look at the AgentChat API.
An agent in AutoGen is an entity defined by the base interface Agent. It has a unique
identifier of the type AgentId, a metadata dictionary of the type AgentMetadata.
In most cases, you can subclass your agents from higher level class RoutedAgent which
enables you to route messages to corresponding message handler specified with
message_handler() decorator and proper type hint for the message variable. An agent
runtime is the execution environment for agents in AutoGen.
Note
Agents are not directly instantiated and managed by application code. Instead, they are
created by the runtime when needed and managed by the runtime.
If you are already familiar with AgentChat, it is important to note that AgentChat’s agents
such as AssistantAgent are created by application and thus not directly managed by the
runtime. To use an AgentChat agent in Core, you need to create a wrapper Core agent that
delegates messages to the AgentChat agent and let the runtime manage the wrapper agent.
Implementing an Agent
To implement an agent, the developer must subclass the RoutedAgent class and implement
a message handler method for each message type the agent is expected to handle using
the message_handler() decorator. For example, the following agent handles a simple
message type MyMessageType and prints the message it receives:
class MyAgent(RoutedAgent):
def __init__(self) -> None:
super().__init__("MyAgent")
@message_handler
async def handle_my_message_type(self, message: MyMessageType, ctx:
MessageContext) -> None:
print(f"{self.id.type} received message: {message.content}")
This agent only handles MyMessageType and messages will be delivered to
handle_my_message_type method. Developers can have multiple message handlers for
different message types by using message_handler() decorator and setting the type hint for
the message variable in the handler function. You can also leverage python typing union for
the message variable in one message handler function if it better suits agent’s logic. See the
next section on message and communication.
class MyAssistant(RoutedAgent):
def __init__(self, name: str) -> None:
super().__init__(name)
model_client = OpenAIChatCompletionClient(model="gpt-4o")
self._delegate = AssistantAgent(name, model_client=model_client)
@message_handler
async def handle_my_message_type(self, message: MyMessageType, ctx:
MessageContext) -> None:
print(f"{self.id.type} received message: {message.content}")
response = await self._delegate.on_messages(
[TextMessage(content=message.content, source="user")], ctx.cancellation_token
)
print(f"{self.id.type} responded: {response.chat_message.content}")
For how to use model client, see the Model Client section.
Since the Core API is unopinionated, you are not required to use the AgentChat API to use
the Core API. You can implement your own agents or use another agent framework.
Registering Agent Type
To make agents available to the runtime, developers can use the register() class method of
the BaseAgent class. The process of registration associates an agent type, which is uniquely
identified by a string, and a factory function that creates an instance of the agent type of the
given class. The factory function is used to allow automatic creation of agent instances when
they are needed.
Agent type (AgentType) is not the same as the agent class. In this example, the agent type
is AgentType("my_agent") or AgentType("my_assistant") and the agent class is the Python
class MyAgent or MyAssistantAgent. The factory function is expected to return an instance
of the agent class on which the register() class method is invoked. Read Agent Identity and
Lifecycles to learn more about agent type and identity.
Note
Different agent types can be registered with factory functions that return the same agent
class. For example, in the factory functions, variations of the constructor parameters can be
used to create different instances of the same agent class.
To register our agent types with the SingleThreadedAgentRuntime, the following code can
be used:
runtime = SingleThreadedAgentRuntime()
await MyAgent.register(runtime, "my_agent", lambda: MyAgent())
await MyAssistant.register(runtime, "my_assistant", lambda: MyAssistant("my_assistant"))
AgentType(type='my_assistant')
Once an agent type is registered, we can send a direct message to an agent instance using
an AgentId. The runtime will create the instance the first time it delivers a message to this
instance.
Because the runtime manages the lifecycle of agents, an AgentId is only used to
communicate with the agent or retrieve its metadata (e.g., description).
runtime.start()
# ... Send messages, publish messages, etc.
await runtime.stop() # This will return immediately but will not cancel
# any in-progress message handling.
You can resume the background task by calling start() again.
For batch scenarios such as running benchmarks for evaluating agents, you may want to
wait for the background task to stop automatically when there are no unprocessed
messages and no agent is handling messages – the batch may considered complete. You
can achieve this by using the stop_when_idle() method:
runtime.start()
# ... Send messages, publish messages, etc.
await runtime.stop_when_idle() # This will block until the runtime is idle.
To close the runtime and release resources, use the close() method:
await runtime.close()
Other runtime implementations will have their own ways of running the runtime.
Message and Communication
An agent in AutoGen core can react to, send, and publish messages, and messages are the
only means through which agents can communicate with each other.
Messages
Messages are serializable objects, they can be defined using:
A dataclass
For example:
@dataclass
class TextMessage:
content: str
source: str
@dataclass
class ImageMessage:
url: str
source: str
Note
Messages are purely data, and should not contain any logic.
Message Handlers
When an agent receives a message the runtime will invoke the agent’s message handler
(on_message()) which should implement the agents message handling logic. If this message
cannot be handled by the agent, the agent should raise a CantHandleException.
The base class BaseAgent provides no message handling logic and implementing the
on_message() method directly is not recommended unless for the advanced use cases.
Developers should start with implementing the RoutedAgent base class which provides built-
in message routing capability.
For example, the following type-routed agent responds to TextMessage and ImageMessage
using different message handlers:
from autogen_core import AgentId, MessageContext, RoutedAgent,
SingleThreadedAgentRuntime, message_handler
class MyAgent(RoutedAgent):
@message_handler
async def on_text_message(self, message: TextMessage, ctx: MessageContext) -> None:
print(f"Hello, {message.source}, you said {message.content}!")
@message_handler
async def on_image_message(self, message: ImageMessage, ctx: MessageContext) ->
None:
print(f"Hello, {message.source}, you sent me {message.url}!")
Create the agent runtime and register the agent type (see Agent and Agent Runtime):
runtime = SingleThreadedAgentRuntime()
await MyAgent.register(runtime, "my_agent", lambda: MyAgent("My Agent"))
AgentType(type='my_agent')
Test this agent with TextMessage and ImageMessage.
runtime.start()
agent_id = AgentId("my_agent", "default")
await runtime.send_message(TextMessage(content="Hello, World!", source="User"),
agent_id)
await runtime.send_message(ImageMessage(url="https://example.com/image.jpg",
source="User"), agent_id)
await runtime.stop_when_idle()
Hello, User, you said Hello, World!!
Hello, User, you sent me https://example.com/image.jpg!
The runtime automatically creates an instance of MyAgent with the agent ID
AgentId("my_agent", "default") when delivering the first message.
The match parameter associates handlers for the same message type to a specific message
– it is secondary to the message type routing. It accepts a callable that takes the message
and MessageContext as arguments, and returns a boolean indicating whether the message
should be handled by the decorated handler. The callable is checked in the alphabetical
order of the handlers.
Here is an example of an agent that routes messages based on the sender agent using the
match parameter:
class RoutedBySenderAgent(RoutedAgent):
@message_handler(match=lambda msg, ctx: msg.source.startswith("user1")) # type:
ignore
async def on_user1_message(self, message: TextMessage, ctx: MessageContext) ->
None:
print(f"Hello from user 1 handler, {message.source}, you said {message.content}!")
Let’s test this agent with messages with different source values:
runtime = SingleThreadedAgentRuntime()
await RoutedBySenderAgent.register(runtime, "my_agent", lambda:
RoutedBySenderAgent("Routed by sender agent"))
runtime.start()
agent_id = AgentId("my_agent", "default")
await runtime.send_message(TextMessage(content="Hello, World!", source="user1-test"),
agent_id)
await runtime.send_message(TextMessage(content="Hello, World!", source="user2-test"),
agent_id)
await runtime.send_message(ImageMessage(url="https://example.com/image.jpg",
source="user1-test"), agent_id)
await runtime.send_message(ImageMessage(url="https://example.com/image.jpg",
source="user2-test"), agent_id)
await runtime.stop_when_idle()
Hello from user 1 handler, user1-test, you said Hello, World!!
Hello from user 2 handler, user2-test, you said Hello, World!!
Hello, user2-test, you sent me https://example.com/image.jpg!
In the above example, the first ImageMessage is not handled because the source field of the
message does not match the handler’s match condition.
Direct Messaging
There are two types of communication in AutoGen core:
Let’s first look at direct messaging. To send a direct message to another agent, within a
message handler use the autogen_core.BaseAgent.send_message() method, from the
runtime use the autogen_core.AgentRuntime.send_message() method. Awaiting calls to
these methods will return the return value of the receiving agent’s message handler. When
the receiving agent’s handler returns None, None will be returned.
Note
If the invoked agent raises an exception while the sender is awaiting, the exception will be
propagated back to the sender.
Request/Response
Direct messaging can be used for request/response scenarios, where the sender expects a
response from the receiver. The receiver can respond to the message by returning a value
from its message handler. You can think of this as a function call between agents.
@dataclass
class Message:
content: str
class InnerAgent(RoutedAgent):
@message_handler
async def on_my_message(self, message: Message, ctx: MessageContext) -> Message:
return Message(content=f"Hello from inner, {message.content}")
class OuterAgent(RoutedAgent):
def __init__(self, description: str, inner_agent_type: str):
super().__init__(description)
self.inner_agent_id = AgentId(inner_agent_type, self.id.key)
@message_handler
async def on_my_message(self, message: Message, ctx: MessageContext) -> None:
print(f"Received message: {message.content}")
# Send a direct message to the inner agent and receves a response.
response = await self.send_message(Message(f"Hello from outer, {message.content}"),
self.inner_agent_id)
print(f"Received inner response: {response.content}")
Upone receving a message, the OuterAgent sends a direct message to the InnerAgent and
receives a message in response.
runtime = SingleThreadedAgentRuntime()
await InnerAgent.register(runtime, "inner_agent", lambda: InnerAgent("InnerAgent"))
await OuterAgent.register(runtime, "outer_agent", lambda: OuterAgent("OuterAgent",
"inner_agent"))
runtime.start()
outer_agent_id = AgentId("outer_agent", "default")
await runtime.send_message(Message(content="Hello, World!"), outer_agent_id)
await runtime.stop_when_idle()
Received message: Hello, World!
Received inner response: Hello from inner, Hello from outer, Hello, World!
Both outputs are produced by the OuterAgent’s message handler, however the second
output is based on the response from the InnerAgent.
Generally speaking, direct messaging is appropriate for scenarios when the sender and
recipient are tightly coupled – they are created together and the sender is linked to a specific
instance of the recipient. For example, an agent executes tool calls by sending direct
messages to an instance of ToolAgent, and uses the responses to form an action-
observation loop.
Broadcast
Broadcast is effectively the publish/subscribe model with topic and subscription. Read Topic
and Subscription to learn the core concepts.
The key difference between direct messaging and broadcast is that broadcast cannot be
used for request/response scenarios. When an agent publishes a message it is one way
only, it cannot receive a response from any other agent, even if a receiving agent’s handler
returns a value.
Note
Note
If an agent publishes a message type for which it is subscribed it will not receive the
message it published. This is to prevent infinite loops.
@type_subscription(topic_type="default")
class ReceivingAgent(RoutedAgent):
@message_handler
async def on_my_message(self, message: Message, ctx: MessageContext) -> None:
print(f"Received a message: {message.content}")
To publish a message from an agent’s handler, use the publish_message() method and
specify a TopicId. This call must still be awaited to allow the runtime to schedule delivery of
the message to all subscribers, but it will always return None. If an agent raises an exception
while handling a published message, this will be logged but will not be propagated back to
the publishing agent.
class BroadcastingAgent(RoutedAgent):
@message_handler
async def on_my_message(self, message: Message, ctx: MessageContext) -> None:
await self.publish_message(
Message("Publishing a message from broadcasting agent!"),
topic_id=TopicId(type="default", source=self.id.key),
)
BroadcastingAgent publishes message to a topic with type "default" and source assigned to
the agent instance’s agent key.
Subscriptions are registered with the agent runtime, either as part of agent type’s registration
or through a separate API method. Here is how we register TypeSubscription for the
receiving agent with the type_subscription() decorator, and for the broadcasting agent
without the decorator.
runtime = SingleThreadedAgentRuntime()
From the output, you can see two messages were received by the receiving agent: one was
published through the runtime, and the other was published by the broadcasting agent.
DefaultTopicId is for creating a topic that uses "default" as the default value for the topic type
and the publishing agent’s key as the default value for the topic source.
default_subscription() is for creating a type subscription that subscribes to the default topic.
We can simplify BroadcastingAgent by using DefaultTopicId and default_subscription().
@default_subscription
class BroadcastingAgentDefaultTopic(RoutedAgent):
@message_handler
async def on_my_message(self, message: Message, ctx: MessageContext) -> None:
# Publish a message to all agents in the same namespace.
await self.publish_message(
Message("Publishing a message from broadcasting agent!"),
topic_id=DefaultTopicId(),
)
When the runtime calls register() to register the agent type, it creates a TypeSubscription
whose topic type uses "default" as the default value and agent type uses the same agent
type that is being registered in the same context.
runtime = SingleThreadedAgentRuntime()
await BroadcastingAgentDefaultTopic.register(
runtime, "broadcasting_agent", lambda: BroadcastingAgentDefaultTopic("Broadcasting
Agent")
)
await ReceivingAgent.register(runtime, "receiving_agent", lambda:
ReceivingAgent("Receiving Agent"))
runtime.start()
await runtime.publish_message(Message("Hello, World! From the runtime!"),
topic_id=DefaultTopicId())
await runtime.stop_when_idle()
Received a message: Hello, World! From the runtime!
Received a message: Publishing a message from broadcasting agent!
Note
If your scenario allows all agents to publish and subscribe to all broadcasted messages, use
DefaultTopicId and default_subscription() to decorate your agent classes.
Logging
AutoGen uses Python’s built-in logging module.
Trace logging: This is used for debugging and is human readable messages to indicate what
is going on. This is intended for a developer to understand what is happening in the code.
The content and format of these logs should not be depended on by other systems.
Name: TRACE_LOGGER_NAME.
Structured logging: This logger emits structured events that can be consumed by other
systems. The content and format of these logs can be depended on by other systems.
Name: EVENT_LOGGER_NAME.
import logging
logging.basicConfig(level=logging.WARNING)
logger = logging.getLogger(TRACE_LOGGER_NAME)
logger.setLevel(logging.DEBUG)
Structured logging
Structured logging allows you to write handling logic that deals with the actual events
including all fields rather than just a formatted string.
For example, if you had defined this custom event and were emitting it. Then you could write
the following handler to receive it.
import logging
from dataclasses import dataclass
@dataclass
class MyEvent:
timestamp: str
message: str
class MyHandler(logging.Handler):
def __init__(self) -> None:
super().__init__()
def emit(self, record: logging.LogRecord) -> None:
try:
# Use the StructuredMessage if the message is an instance of it
if isinstance(record.msg, MyEvent):
print(f"Timestamp: {record.msg.timestamp}, Message: {record.msg.message}")
except Exception:
self.handleError(record)
And this is how you could use it:
logger = logging.getLogger(EVENT_LOGGER_NAME)
logger.setLevel(logging.INFO)
my_handler = MyHandler()
logger.handlers = [my_handler]
Emitting logs
These two names are the root loggers for these types. Code that emits logs should use a
child logger of these loggers. For example, if you are writing a module my_module and you
want to emit trace logs, you should use the logger named:
import logging
import logging
from dataclasses import dataclass
from autogen_core import EVENT_LOGGER_NAME
@dataclass
class MyEvent:
timestamp: str
message: str
Clean instrumentation
If you do not have open telemetry set up in your application, you can follow these steps to
instrument your application.
return tracer_provider
Now you can send the trace_provider when creating your runtime:
The system that provides this is called “component configuration”. In AutoGen, a component
is simply something that can be created from a config object and itself can be dumped to a
config object. In this way, you can define a component in code and then get the config object
from it.
This system is generic and allows for components defined outside of AutoGen itself (such as
extensions) to be configured in the same way.
Component configuration should be thought of as the blueprint for an object, and can be
stamped out many times to create many instances of the same configured object.
Usage
If you have a component in Python and want to get the config for it, simply call
dump_component() on it. The resulting object can be passed back into load_component() to
get the component back.
config = {
"provider": "openai_chat_completion_client",
"config": {"model": "gpt-4o"},
}
client = ChatCompletionClient.load_component(config)
Creating a component class
To add component functionality to a given class:
For example:
from autogen_core import Component, ComponentBase
from pydantic import BaseModel
class Config(BaseModel):
value: str
@classmethod
def _from_config(cls, config: Config) -> "MyComponent":
return cls(value=config.value)
Secrets
If a field of a config object is a secret value, it should be marked using SecretStr, this will
ensure that the value will not be dumped to the config object.
For example:
class ClientConfig(BaseModel):
endpoint: str
api_key: SecretStr