18. Multi-Agent Systems — The Supervisor Pattern

Think back to the customer-service agent we built in Chapter 16. It had exactly one tool — order lookup — so when a customer asked about an order, it reported the shipping status. With only one tool, there was simply no way for it to pick the wrong one.

Now suppose we grow that agent into a real production service. Order lookup alone isn't enough. We'd need order cancellation, shipment tracking, address changes, exchange requests, return requests, refund-eligibility checks, refund processing, inventory checks, coupon issuance, points lookup, support-ticket creation, and more. The mechanics are simple: keep adding tools and appending business rules to the system prompt.

But as the tools and rules pile up, three problems emerge.

• Tool selection gets less accurate. On every turn the model reads all the tool descriptions and decides which one to call. As you add tools that take the same inputs and overlap in purpose — like exchange request and return request — the odds of picking the wrong one climb.

• The context fills with information you don't currently need. Even while processing a refund, the inventory-check tool's schema, the coupon-issuance rules, the exchange-request procedure, and everything else ride along on every call. You're shipping the full set of tool schemas and every domain's business rules each time. When the context is crowded with material irrelevant to the current task, the parts that matter get buried and accuracy suffers.

• It gets hard to change. Adjusting a single refund rule means editing a prompt where every domain's rules are tangled together, and you can't be sure the change won't ripple into exchanges or shipping. If different domains have different owners, the problem only grows.

This chapter teaches one way to deal with this. Instead of piling more tools and rules onto a single agent, we split the work into one agent per domain. Continuing the customer-service scenario from Chapter 16, we'll build a team made of an agent that only handles order lookups and an agent that only handles refunds. Each has its own tools and its own prompt. Then we add one more agent to direct them — this director is called the supervisor. A setup with several agents like this is a multi-agent system, and the arrangement where a supervisor commands the rest is the supervisor pattern.

Multi-agent systems do come at a cost. Because the supervisor has to decide which worker to delegate to on every step, there are more LLM calls, and that means more latency and more expense. So if you don't have that many tools and rules, there's no need to reach for a multi-agent system at all.

Here's the plan. In 18.1 we look at what a multi-agent system is and how it works. In 18.2 we build the worker agents. In 18.3 we build a supervisor by hand with LangGraph's Command object. In 18.4 we wrap the workers as tools and rebuild the same team with far less code.

18.1) Understanding Multi-Agent Systems

18.1.1) What a Multi-Agent System Is

Let's design an agent that can handle the request, "My order never arrived — if something's wrong, refund it." We could build it with everything we've learned so far: attach an order-lookup tool and a refund tool to one agent, and let the agent loop until the job is done. A single agent looks up the order, confirms the delivery failed, sees that result, requests the refund, and writes the final reply.

A multi-agent system is a structure where several agents divide this work between them. You split agents by domain and place a supervisor above them, and each agent holds only the tools it needs. The agent we just sketched, for instance, splits naturally into an order-lookup agent and a refund agent. The supervisor first calls the order-lookup agent to check the shipping status; when it hears back that delivery failed, it calls the refund agent to process the refund; then it gathers both results to answer the customer. What used to be one agent calling tools in sequence has become a supervisor calling agents in sequence.

If there were only two tools, there'd be no reason to split like this. One agent would be enough, and adding a supervisor would only add LLM calls. But as we saw in the introduction, once there are many tools, an agent can pick the wrong one, its context fills with information unrelated to the task at hand, and its prompt becomes hard to change.

Splitting solves these problems. The order-lookup agent sees only a few order-related tools, so choosing from a list of dozens shrinks to choosing among a handful. Its prompt holds only order-lookup rules, so refund policy, coupon conditions, and other matters unrelated to order lookup don't fill its context. And when you need to change a refund rule, you touch only the refund agent, so the change doesn't reach the others.

The order-lookup agent and the refund agent here are nothing special. They're the same kind of agent you built in Chapter 16. You create them by passing a model, tools, and a prompt to create_agent, and you call them with invoke. They just cover less ground.

So what does the supervisor do? The order-lookup agent and the refund agent don't know the other exists. Each just does its own job; neither knows who should go first. In the example above, calling order lookup first and — only after seeing its result — calling refund was the supervisor's judgment.

Who calls whom, and when. This is what we call orchestration.

18.1.2) The Supervisor Pattern

The supervisor pattern is a structure in which a single central supervisor orchestrates several worker agents. It follows these rules:

• The supervisor never does the work itself. It doesn't look up orders or process refunds. It only decides who to hand off to, then gathers the returned results into a final answer.
• Workers never call each other. The order-lookup agent never calls the refund agent directly. Every path runs through the supervisor.
• Only the supervisor talks to the customer. Workers report to the supervisor, not to the customer.

So how does the supervisor decide which agent to call? The LLM decides. The supervisor reads the entire conversation so far and judges. If it doesn't yet know the order status, it calls the order-lookup agent; once it's confirmed that delivery failed and a refund is needed, it calls the refund agent.

The supervisor repeats this judgment until the user's request is complete. It calls an agent, receives a report, re-reads the conversation now that the report has been added, and decides which agent to call next.

This loop has the same structure as the one you built in Chapter 14.

• Think — read the conversation so far and decide which agent to call.
• Act — run the chosen agent.
• Observe — take the agent's report and add it to the conversation.

In Chapter 14 you called tools; here you call agents. That's the only thing that changes.

Notice how the arrows loop back to the supervisor. When a worker finishes, it reports to the supervisor, and the supervisor reads that report and decides the next move.

The supervisor is what ends the loop. Once it judges that the customer's request is fully handled, it produces the final answer and stops.

18.2) Building the Worker Agents

Let's build the two worker agents we designed in 18.1: an order-lookup worker that checks order status, and a refund worker that processes refunds. We'll build the supervisor in 18.3.

A worker agent is just the ordinary agent you built in Chapter 16. We'll build them quickly with create_agent.

First, let's set up the order data the two workers will share.

ORDERS = {
    "12345": {"item": "Wireless Earbuds", "amount": 89,
              "status": "in_transit", "status_text": "In transit (arriving tomorrow)"},
    "67890": {"item": "Mechanical Keyboard", "amount": 129,
              "status": "delivered", "status_text": "Delivered"},
    "24680": {"item": "Noise-Cancelling Headphones", "amount": 249,
              "status": "delivery_failed", "status_text": "Delivery failed (returned — recipient not found)"},
}

The order-lookup worker has just one tool.

from langchain.tools import tool
from langchain.agents import create_agent
 
@tool
def get_order_status(order_id: str) -> str:
    """Look up the item, amount paid, and shipping status for an order number."""
    order = ORDERS.get(order_id)
    if order is None:
        return f"Order {order_id} not found."
 
    return (f"Order {order_id}: {order['item']}, "
            f"${order['amount']:,}, status: {order['status_text']}")
 
 
order_agent = create_agent(
    name="order_expert",
    model="openai:gpt-5.4-mini",
    tools=[get_order_status],
    system_prompt=(
        "You are an order-lookup specialist. Look up the order status and answer.\n"
        "Include the order number, item, amount paid, and shipping status in your final answer.\n"
        "Do not judge whether a refund is warranted or mention refunds in any way. Your role is order lookup and status reporting only."
    ),
)

The refund worker has two tools: one that decides whether an order is refund-eligible, and one that actually processes the refund.

@tool
def check_refund_eligibility(order_id: str) -> str:
    """Determine whether an order is eligible for a refund. Only failed-delivery orders qualify."""
    order = ORDERS.get(order_id)
    if order is None:
        return f"Order {order_id} not found."
 
    if order["status"] == "delivery_failed":
        return f"Order {order_id} is eligible for a refund (reason: failed delivery)."
 
    return f"Order {order_id} is not eligible for a refund (current status: {order['status_text']})."
 
 
@tool
def issue_refund(order_id: str) -> str:
    """Process a refund. Always confirm eligibility with check_refund_eligibility before calling this."""
    order = ORDERS.get(order_id)
    if order is None:
        return f"Order {order_id} not found."
 
    return (f"Refund complete: ${order['amount']:,} for order {order_id} "
            f"will be refunded within 3–5 business days. (approval number: RF-{order_id})")
 
 
refund_agent = create_agent(
    name="refund_expert",
    model="openai:gpt-5.4-mini",
    tools=[check_refund_eligibility, issue_refund],
    system_prompt=(
        "You are a refund-processing specialist.\n"
        "Always confirm eligibility with check_refund_eligibility first, and only "
        "then call issue_refund.\n"
        "If you processed a refund, include the amount and the approval number in your final answer.\n"
        "If the order isn't eligible, don't process it — report the reason instead."
    ),
)

We gave both workers a name. This name is what create_supervisor in 18.5 uses as the node name and the handoff-tool name.

Four Things a Worker's System Prompt Needs

A worker's system prompt is built from four elements — principles Anthropic distilled while building its own multi-agent research system. When these are weak, workers duplicate work, leave tasks undone, or fail to find the information they need.

Role fixes, in one sentence, who this worker is. Pinning down its identity with "You are an order-lookup specialist" keeps the model focused on its own job and less likely to wander into someone else's.

Tool guidance. Write this when there's something the tool schema alone can't convey — the order and conditions under which tools should be used, for example. If there's nothing to add beyond the schema, you can leave it out.

Output format and task boundary matter a great deal in a multi-agent setting.

Output format. A worker's final answer isn't a reply to the customer — it's a report submitted to the supervisor. Anything not written there never reaches the supervisor. If a worker looks up an amount with a tool but leaves it out of its final answer, the supervisor has no way to know it.

Task boundary. This is the scope that defines how far a worker may go and what it must not do. The order-lookup worker should only look up — never refund. That's why we didn't give it a refund tool. But withholding the tool isn't enough on its own, because the model can still say "Delivery failed, so I'll issue a refund for you" with no tool at all. If that sentence reaches the supervisor, the supervisor may assume a refund is already underway and never call the refund worker. So the prompt also says, "Do not judge whether a refund is warranted or mention refunds in any way," blocking it from even bringing up refunds in words.

Choosing Models

We're using gpt-5.4-mini for the workers and gpt-5.4 for the supervisor. A worker does the simple job of calling a few tools in a set order, so a small model is plenty. The supervisor has to read the whole conversation and judge who to call next, so it needs a larger model. Being able to pick a model per agent, matched to the difficulty of its job, is another benefit of splitting.

Now we can add the supervisor.

18.3) Building the Supervisor by Hand

Let's build a supervisor by hand. In practice you'll mostly use the approach where the framework handles this for you (covered in the next section), but to understand what's happening under the hood, you need to build it yourself once.

As we saw in 18.1, what the supervisor does is a single loop: call a worker, read the report, and decide who to call next — or whether to stop — over and over.

18.3.1) Handoffs and Command

For this loop to turn, control has to pass back and forth between the supervisor and the workers. The supervisor hands off control — "this worker goes next" — and when the worker finishes, it hands control back to the supervisor. This passing of control from one node to another is called a handoff.

A handoff needs two pieces of information: where to go (the destination) and what to pass along (the payload). The destination is always required; the payload is included only when there's something to pass. In LangGraph, a node specifies both by returning a Command.

from typing import Literal
from langgraph.graph import MessagesState
from langgraph.types import Command
 
def some_node(state: MessagesState) -> Command[Literal["refund_expert_proxy"]]:
    return Command(
        goto="refund_expert_proxy",             # where: the node to run next
        update={"messages": [...]},             # what: the worker's report (the payload added to State)
    )

goto is the destination; update is the payload. The return type hint Command[Literal["refund_expert_proxy"]] lists, up front, the destinations this node can go to. We'll see how this Command is actually used in the next section, where we build the supervisor node and the worker proxies.

18.3.2) Building the Supervisor Loop

The structure itself is simple. We make one supervisor node and as many worker proxies as we need. A worker proxy is a node that calls its assigned worker agent on the supervisor's behalf. The entry point is the supervisor, and each worker proxy, once done, moves back to the supervisor — forming the loop. The Command each node returns is what decides where control goes next.

Solid lines are handoffs between nodes (goto); dotted lines are a worker proxy calling its worker agent (invoke). The supervisor hands off to a worker proxy, and the worker proxy, once finished, returns to the supervisor. When the supervisor decides FINISH, it exits to END.

Now let's turn this picture into code. First, the Route class. Route is the schema for receiving the supervisor's answer as a structured response when we ask the LLM which worker proxy to call next. If the LLM answered in free-form natural language, it would be hard to tell which worker proxy to run. Route holds which worker proxy to call next (next) and the reason it decided that way (reason).

from typing import Literal
from pydantic import BaseModel, Field
 
class Route(BaseModel):
    reason: str = Field(description="The reason for this decision.")
    next: Literal["order_expert_proxy", "refund_expert_proxy", "FINISH"] = Field(
        description="The worker node to run next. FINISH if the request is fully handled."
    )

There's a reason reason is declared before next. Structured output is generated in the order the fields appear in the schema, so with reason first, the LLM writes its reasoning before choosing a worker. Reasoning first and deciding second yields a better choice. Flip the order — put next first — and the LLM picks a worker before it has reasoned at all, then backfills a rationale to fit a choice it may already have gotten wrong.

Next, the supervisor node.

from typing import Literal
from langgraph.graph import MessagesState, StateGraph, START, END
from langgraph.types import Command
from langchain_openai import ChatOpenAI
from langchain_core.messages import HumanMessage
 
supervisor_llm = ChatOpenAI(model="gpt-5.4")
 
SUPERVISOR_PROMPT = (
    "You are the supervisor of a customer-service team. You manage two workers:\n"
    "- order_expert_proxy: looks up order status.\n"
    "- refund_expert_proxy: checks refund eligibility and processes refunds.\n"
    "To decide whether a refund is needed, you must check the order status first.\n"
    "Assign to one worker at a time, and respond with FINISH once the request is fully handled."
)
 
 
def supervisor(
    state: MessagesState,
) -> Command[Literal["order_expert_proxy", "refund_expert_proxy", "__end__"]]:
    messages = [{"role": "system", "content": SUPERVISOR_PROMPT}, *state["messages"]]
    decision = supervisor_llm.with_structured_output(Route).invoke(messages)
    print(f"[supervisor] → {decision.next} ({decision.reason})")
 
    if decision.next == "FINISH":
        final = supervisor_llm.invoke(
            [{"role": "system", "content": "Based on the conversation so far, write a reply to the customer."},
             *state["messages"]]
        )
        return Command(goto=END, update={"messages": [final]})   # hand off to END
 
    return Command(goto=decision.next)                            # hand off to the worker proxy

Until now, nodes returned only the changed State. But the supervisor node returns a Command. When a node returns a Command, LangGraph does two things: it applies the contents of update to the State, and it runs the node named in goto next. In the code above, we put the worker-proxy name pulled from decision.next into goto, so the node the LLM chose — decision.next — is what runs.

Next, the worker proxies. A worker proxy does its job and returns control to the supervisor — which is why it hands off with goto="supervisor".

A worker proxy's job is simple: call its worker agent with invoke.

def order_expert_proxy(state: MessagesState) -> Command[Literal["supervisor"]]:
    result = order_agent.invoke(state)
    last = result["messages"][-1]
    return Command(
        goto="supervisor",
        update={"messages": [HumanMessage(content=last.content, name="order_expert")]},
    )
 
 
def refund_expert_proxy(state: MessagesState) -> Command[Literal["supervisor"]]:
    result = refund_agent.invoke(state)
    last = result["messages"][-1]
    return Command(
        goto="supervisor",
        update={"messages": [HumanMessage(content=last.content, name="refund_expert")]},
    )

Notice that we pull only the worker's final message and pass that to the supervisor. The supervisor only needs the conclusion; it doesn't need to know how many times the worker called tools internally.

18.3.3) Wiring and Running the Graph

We register the three nodes and connect only the entry point to the supervisor. Every other move is decided by each node's Command, so no further edges are needed.

builder = StateGraph(MessagesState)
builder.add_node("supervisor", supervisor)
builder.add_node("order_expert_proxy", order_expert_proxy)
builder.add_node("refund_expert_proxy", refund_expert_proxy)
builder.add_edge(START, "supervisor")
 
team = builder.compile()
 
result = team.invoke(
    {"messages": [HumanMessage(
        content="Order 24680 still hasn't arrived. If something's wrong, please refund it."
    )]},
    config={"recursion_limit": 15},
)

Output:

[supervisor] → order_expert_proxy (Need to check the order status before deciding on a refund.)
[supervisor] → refund_expert_proxy (Delivery failed is confirmed, so check refund eligibility and process it.)
[supervisor] → FINISH (Order check and refund processing are both complete.)

The supervisor first routed to order lookup. When the report came back that delivery had failed, it routed to refund, and when the refund-complete report arrived, it finished. It chose each next destination by reading the previous agent's report.

The worker proxy passes the entire shared State to its worker via invoke(state), so each worker sees the whole conversation so far. With just two workers this is fine, but as workers and conversation grow, each worker ends up reading messages that have nothing to do with its own job. We'll solve this differently in the next section.

18.4) Delegating to Workers Through Tools

Having built the supervisor's internals by hand in 18.3, let's now rebuild the same team the way that's recommended for real projects. This approach needs no new API. You turn each worker into a tool with @tool, and give those tools to a supervisor agent. We'll build the supervisor agent simply with create_agent.

The key idea fits in one sentence: the supervisor is itself just an agent, and each worker becomes a tool the supervisor calls.

Seen this way, the supervisor has the same structure as the tool-calling agent you built in Chapter 16. It just holds high-level tools that call agents, instead of low-level tools like get_order_status.

18.4.1) Wrapping Workers as Tools

We use the order_agent and refund_agent from 18.2 unchanged. All we do is wrap each in a @tool function.

from langchain.tools import tool
 
# order_agent and refund_agent are the workers from 18.2
 
@tool
def lookup_order(request: str) -> str:
    """Look up an order's item, amount paid, and shipping status. Use this when you need to know an order's status.
 
    Input: a natural-language lookup request (e.g., 'Tell me the shipping status of order 24680').
    """
    print("[tool call] lookup_order")
    print(f"    request: {request}")
 
    result = order_agent.invoke({"messages": [{"role": "user", "content": request}]})
    return result["messages"][-1].content
 
 
@tool
def handle_refund(request: str) -> str:
    """Check refund eligibility and process a refund. Use this when the customer wants a refund and you've already confirmed the order status.
 
    Input: a natural-language refund request. Include the order number and the shipping status confirmed by lookup.
    (e.g., 'Order 24680 is in failed-delivery status. Process a refund if eligible.')
    """
    print("[tool call] handle_refund")
    print(f"    request: {request}")
 
    result = refund_agent.invoke({"messages": [{"role": "user", "content": request}]})
    return result["messages"][-1].content

Three things changed.

• The tool descriptions replace the routing logic. In 18.3 we wrote SUPERVISOR_PROMPT and the Route schema by hand to tell the supervisor its list of workers and its choices. Here the tool docstrings do that job. The supervisor's LLM reads the tool descriptions and decides when to call what.

• Each worker starts from a clean context. Set side by side with 18.3, the difference is clear.

  python

  # 18.3 (manual graph): passes the entire shared State
  result = refund_agent.invoke(state)
   
  # 18.4 (tool delegation): passes only the task description the supervisor wrote
  result = refund_agent.invoke({"messages": [{"role": "user", "content": request}]})

  In 18.4 the refund worker receives just one sentence describing its task. It never sees the customer's original wording, the supervisor's reasoning, or another worker's tool-call history. Even with ten workers and a hundred turns of conversation, each worker's context is still just that one task description.

• In exchange, the supervisor takes on the job of passing information. Because a worker can't see the conversation history, whatever it needs has to be packed into the request string by the supervisor. That's why handle_refund's docstring spells out, "Include the order number and the shipping status confirmed by lookup." Without that instruction, the supervisor might pass only "Process a refund", leaving the refund worker unsure which order it's even about.

18.4.2) Assembling and Running the Supervisor

In this approach the supervisor, too, is just an agent. No Command, no Route schema needed.

from langchain.agents import create_agent
from langchain_core.messages import HumanMessage
 
TOOL_SUPERVISOR_PROMPT = (
    "You are the supervisor of a customer-service team.\n"
    "To decide whether a refund is needed, you must check the order status first.\n"
    "Don't do the work yourself — delegate to the workers.\n"
    "The workers cannot see this conversation. When you delegate, put everything they need into the request.\n"
    "When all the work is done, synthesize the workers' results into a reply to the customer."
)
 
supervisor_agent = create_agent(
    model="openai:gpt-5.4",
    tools=[lookup_order, handle_refund],
    system_prompt=TOOL_SUPERVISOR_PROMPT,
)
 
result = supervisor_agent.invoke(
    {"messages": [HumanMessage(
        content="Order 24680 still hasn't arrived. If something's wrong, please refund it."
    )]}
)
 
print("\n\n[final response]")
print(result["messages"][-1].content)

The end result is the same as in 18.3.

[tool call] lookup_order
    request: The customer says order 24680 hasn't arrived yet. To decide whether a refund is
             needed, please give me the item, amount paid, and current shipping status for order 24680.
[tool call] handle_refund
    request: Order 24680 is a Noise-Cancelling Headphones, $249, and its shipping status is
             confirmed as 'Delivery failed (returned — recipient not found)'. The customer is
             requesting a refund, so check eligibility and process it if eligible.
 
 
[final response]
I checked, and order 24680 was in failed-delivery status (returned — recipient not found).
 
It qualified for a refund, and I've completed the refund.
- Item: Noise-Cancelling Headphones
- Refund amount: $249
- Refund approval number: RF-24680
 
Depending on your payment method, it usually takes a few business days for the refund to post.

But look at the tool calls the supervisor made along the way — that's where the difference from 18.3 shows. Look at handle_refund's request. The supervisor summarized the earlier lookup result and wrote the task description itself. The refund worker receives only this one sentence. Where 18.3 handed the worker the entire conversation and let it dig through, here the supervisor picks out just what's needed and passes that along.

18.4.3) Adding Memory to the Supervisor with a Checkpointer

Saying the supervisor is an ordinary agent means the checkpointing you learned in Chapter 17 works on it as-is.

from langgraph.checkpoint.memory import InMemorySaver
 
supervisor_agent = create_agent(
    model="openai:gpt-5.4",
    tools=[lookup_order, handle_refund],
    system_prompt=TOOL_SUPERVISOR_PROMPT,
    checkpointer=InMemorySaver(),      # checkpointer goes on the top-level agent only
)
 
config = {"configurable": {"thread_id": "cs-1"}}
 
supervisor_agent.invoke(
    {"messages": [HumanMessage(content="Where's my order 12345?")]},
    config,
)
follow_up = supervisor_agent.invoke(
    {"messages": [HumanMessage(content="How much was that?")]},
    config,
)
print(follow_up["messages"][-1].content)

Output:

Your order 12345, the Wireless Earbuds, was $89.

The supervisor correctly reads that in the follow-up as order 12345 from the previous turn. The checkpointer works on the supervisor just the same.

Don't attach a checkpointer to the worker agents (order_agent, refund_agent). If you do, a worker will carry the results of its previous call into the current one, which can interfere with the task at hand. Without one, a worker runs on nothing but the supervisor's request. For subagents, this is the recommended default.

18.5) create_supervisor: The Form You'll Meet in Legacy Code

In existing codebases and older tutorials, you'll come across the create_supervisor helper from the langgraph-supervisor package. Give it a list of agents and a prompt, and it builds the whole supervisor graph for you.

from langgraph_supervisor import create_supervisor
from langchain_openai import ChatOpenAI
 
workflow = create_supervisor(
    agents=[order_agent, refund_agent],   # each agent must have a name set
    model=ChatOpenAI(model="gpt-5.4"),
    prompt="Assign order checks to order_expert and refunds to refund_expert.",
)
app = workflow.compile()

create_supervisor is a helper that assembles a supervisor team in a single function call (the handoff approach from 18.3). Don't use it in new projects, though. It's legacy that LangChain no longer recommends, and internally it depends on create_react_agent, which was deprecated in v1 (removal is planned for v2). LangChain recommends the tool-based supervisor you learned in 18.4.

In this chapter we took an agent that had lived inside a single graph and expanded it into a team — split by domain, orchestrated by a supervisor. We built the same team three ways: the manual Command graph in 18.3, the tool-delegation approach in 18.4, and the legacy create_supervisor helper in 18.5. For real work, make the tool-delegation approach your default.