5. Agent Preview — The "Aha!" Moment

Up to this point, you've built systems where you control the flow. You write the prompt, call the LLM, and process the response. The LLM is powerful, but it's still just following your instructions.

This chapter introduces a fundamental shift: what if the LLM decides what happens next?

Why This Chapter Exists

The Problem: If you continue building chatbots through Chapter 11 without this preview, you'll likely think "agents are just better chatbots."

The Truth: Agents are fundamentally different. They make decisions that control your program, not just generate text.

The Solution: This chapter shows you where we're going, so when you build pipelines and state management in the next chapters, you'll understand why each piece matters.

What You'll Learn (And What You Won't)

This chapter (Chapter 5):

• A minimal 15-minute preview of agentic thinking
• How LLM output can trigger different code paths
• The conceptual difference between routing and agent loops

Later chapters (Starting Chapter 12):

• Real agent systems with tools, loops, and self-correction
• Production-ready implementations
• Advanced patterns like multi-agent coordination

Why the gap? Before building agents, you need foundations: prompt engineering (Ch 4), execution pipelines (Ch 6), structured output (Ch 7), state management (Ch 8), and knowledge integration (Ch 9-11).

A note on learning approach: This chapter focuses on concepts, not implementation details. The actual design and development techniques come in Chapter 12 and beyond. Don't worry if you don't know how to implement everything you see here—that's intentional. Understanding what an agent is conceptually is enough for now.

Let's begin.

5.1) From Chatbot to Decision Maker

The Concept: Instead of answering the question, the LLM decides which tool to use

In Chapters 3 and 4, you built chat systems where the LLM generates text responses. The flow was straightforward:

1. User asks a question
2. LLM generates an answer
3. You display the answer

This is a static chain: the flow is predetermined. The LLM's only job is to produce text.

Now let's change the type of request. A user asks: "What's 847 × 923?"

Your chatbot from Chapter 3 would attempt to answer this, but LLMs don't actually compute—they predict plausible-looking tokens. For "2 + 2", the answer "4" appears so often in training data that the LLM gets it right. But for "847 × 923"—a calculation the LLM has never seen—it will generate something that looks like a number but is likely wrong.

What you really want:

1. LLM recognizes: "This is a math problem"
2. LLM decides: "Use the calculator tool, not my token prediction"
3. Python executes: 847 × 923 = 781,781
4. System returns: The correct answer

The LLM's job isn't to calculate—it's to route to the tool that can.

Here's the key insight: the LLM doesn't need to solve the math—it needs to decide to use a calculator.

This is the shift from chatbot to decision maker. The LLM examines the request and routes it to the appropriate tool. It's making a decision about program flow, not just generating text.

Let's visualize this difference:

In the static chain, the LLM tries to answer everything directly. In the dynamic routing system, the LLM routes the request to the right tool.

The Visual: Compare a "Linear Chain" (Static) vs. a "Router" (Dynamic)

Here's how a static chain handles any user request:

The LLM tries to answer everything directly. Whether you ask "What is the capital of France?" or "What's 15% of 240?", the LLM generates a text response. It might get the capital right (it's seen "Paris" many times in training data), but it will likely miscalculate the percentage.

The problem: The LLM uses the same approach for all questions—token prediction—even when better tools exist.

Now let's introduce a system that can handle different types of requests differently:

1. Factual questions: "What is the capital of France?" → Use a search tool
2. Math problems: "What's 15% of 240?" → Use a calculator
3. Conversational: "How are you?" → LLM responds directly

Here's a dynamic router that makes this possible:

The LLM examines the input and chooses a path based on what type of request it is. This is routing logic, and the LLM acts as the router.

The key difference: Instead of always generating text, the LLM now decides how to handle each request—by routing it to the appropriate tool.

Key Takeaway: From Generating Answers to Choosing Actions

This is the fundamental shift in agentic systems:

Traditional LLM usage: You ask a question, the LLM writes an answer.

Agentic LLM usage: You ask a question, the LLM decides what to do, and your code executes that decision.

The LLM's output is no longer just text for the user—it's instructions for your program.

Think of it like this: in a traditional chatbot, the LLM is an employee who answers customer questions. In an agentic system, the LLM is a manager who decides which department should handle each request.

This decision-making capability is the essence of agentic systems. The LLM doesn't just respond—it controls what your program does next. Based on the input, it can trigger a calculator, call a search API, or respond directly. The program's behavior changes based on the LLM's decision.

But how do we actually implement this? Let's see the minimal code that makes this work.

5.2) The Minimal Execution

The Trigger: A system prompt that forces structured output

To make the LLM act as a router, we need to constrain its output. Instead of generating a full answer, we want it to output a structured decision—a JSON object that tells our code both what to do and what data to use.

Here's a system prompt that does this:

from langchain_openai import ChatOpenAI
from langchain_core.messages import SystemMessage, HumanMessage
import json
 
llm = ChatOpenAI(model="gpt-4o-mini")
 
system_prompt = """You are a routing assistant. Analyze the user's request and respond with JSON.
 
Output format:
{
  "action": "CALC" | "SEARCH" | "CHAT",
  "input": "cleaned input for the tool"
}
 
Examples:
User: "Calculate 15 times 7"
Output: {"action": "CALC", "input": "15 * 7"}
 
User: "What's the capital of Japan?"
Output: {"action": "SEARCH", "input": "capital of Japan"}
 
User: "Hello there!"
Output: {"action": "CHAT", "input": "Hello there!"}
 
Extract the essential information and format it for the appropriate tool.
Only output valid JSON, nothing else."""
 
def get_routing_decision(user_input: str) -> dict:
    messages = [
        SystemMessage(content=system_prompt),
        HumanMessage(content=user_input)
    ]
    response = llm.invoke(messages)
    
    # Parse JSON response
    try:
        decision = json.loads(response.content.strip())
        return decision
    except json.JSONDecodeError:
        # Fallback if LLM doesn't output valid JSON
        return {"action": "CHAT", "input": user_input}
 
# Test it with DIFFERENT inputs than the examples
print(get_routing_decision("What's 25 * 48?"))  
# Output: {'action': 'CALC', 'input': '25 * 48'}
 
print(get_routing_decision("Who wrote Hamlet?"))  
# Output: {'action': 'SEARCH', 'input': 'author of Hamlet'}
 
print(get_routing_decision("How are you today?"))  
# Output: {'action': 'CHAT', 'input': 'How are you today?'}

This prompt is deliberately restrictive. We're not asking the LLM to be creative—we're asking it to classify the input and extract the relevant information in a structured format.

Notice what's happening here:

1. The LLM reads the user's question
2. It determines the intent (calculation, factual search, conversation)
3. It extracts and cleans the essential information
4. It outputs a JSON object with both the action and cleaned input
5. Our code receives this structured data and routes accordingly

The LLM isn't answering the question—it's deciding what should answer the question.

The Bridge: Connecting LLM Decisions to Tool Execution

Now we need to connect the LLM's decision to actual tool execution. Here's the minimal bridge:

def safe_calculator(expression: str) -> float:
    try:
        # Use eval with restricted namespace for safety
        # Note: eval() has security risks.
        # In production, use a math parsing library like sympy or ast-based evaluation instead.
        result = eval(expression, {"__builtins__": {}}, {})
        return float(result)
    except:
        raise ValueError(f"Could not calculate: {expression}")
 
def search(query: str) -> str:
    """Placeholder search function"""
    # In reality, this would call a search API
    # Now receives cleaned query like "author of Hamlet"
    return f"Search results for: {query}"
 
def route_and_execute(user_input: str) -> str:
    """Get LLM decision and execute the appropriate tool"""
    decision = get_routing_decision(user_input)
    
    action = decision["action"]
    tool_input = decision["input"]  # LLM-cleaned input
    
    if action == "CALC":
        try:
            result = safe_calculator(tool_input)
            return f"Calculation result: {result}"
        except ValueError as e:
            return f"Could not perform calculation: {e}"
    
    elif action == "SEARCH":
        result = search(tool_input)
        return result
    
    else:  # CHAT
        # For conversational queries, let the LLM respond directly
        response = llm.invoke([HumanMessage(content=tool_input)])
        return response.content
 
# Test the complete flow
print(route_and_execute("What's 25 * 48?"))
# LLM extracts "25 * 48" → calculator receives clean input
# Output: Calculation result: 1200.0
 
print(route_and_execute("Who wrote Hamlet?"))
# LLM reformats to "author of Hamlet" → better search query
# Output: Search results for: author of Hamlet
 
print(route_and_execute("How are you today?"))
# Output: I'm doing well, thank you for asking!

This is the minimal routing pattern in its simplest form:

1. LLM Decision: Get routing decision from LLM
2. Code Execution: Python code checks the decision
3. Tool Invocation: Appropriate function is called
4. Result Return: Output is formatted and returned

The critical insight: The LLM's output is no longer text for the user—it's structured data that controls your program's behavior. The action field tells your code which path to take, and the input field provides the cleaned data for that tool. This JSON response becomes executable logic.

This is fundamentally different from a chatbot. In a chatbot, the LLM's output goes directly to the user. Here, the LLM's output goes to your code, which then decides what to execute.

What we built in this section is a router—a precursor to full agents. It demonstrates the fundamental principle (LLM controls flow), but lacks the iteration and self-correction that define true agent loops.

This simple routing system has a major limitation: it can't correct its own mistakes. Let's explore why that matters and what comes next.

5.3) Understanding the Terminology: Router vs Agent

What we just built is a router. But how does it compare to full agents? Let's establish clear definitions:

Router (what we just built)

• Makes one classification decision per request
• Chooses which tool/path to execute
• Executes once and returns
• No iteration, no state, no self-correction
• Example: Email classifier, intent detector, tool selector

Tool-calling assistant (Chapter 13)

• LLM can invoke tools directly via function calling API
• Still typically single-turn (one request → one response)
• More sophisticated than routing, but not necessarily iterative
• Example: "Search for weather and summarize" in one call

Agent (full loop) (Chapters 14-17)

• Iterates through think → act → observe cycles
• Carries state between iterations
• Can revise decisions based on results
• Implements self-correction
• Example: Debugging assistant that tries fixes until code works

Agentic system (umbrella term)

• Any system where LLM output influences control flow
• Includes routers, tool-calling assistants, and full agents
• "Agentic" describes the property; "agent" describes a specific architecture
• Example: All of the above exhibit agentic behavior

What we built in 5.2 is a router—it makes one decision and executes it. This works well for straightforward classification tasks, but it can't handle situations that require multiple steps, verification, or course correction. If the initial decision is suboptimal or if the task turns out to be more complex than expected, the router has no mechanism to adapt.

Agents solve this limitation by introducing iteration. They can observe the result of an action, reconsider their approach, and try again. This makes them suitable for complex, multi-step tasks where the path to the solution isn't clear from the start.

In Chapters 14-17, you'll learn how to build these iterative agent loops.

Note: Recently, reasoning models like o1 and o3 can handle some multi-step tasks internally, reducing the need for explicit loops in certain scenarios. We'll explore when to use loops vs. reasoning models as you build real agents in Part IV.

What you should understand now:

1. Agentic systems start with control flow: The LLM's output determines what your code does next
2. Routing is the simplest form: One decision, one tool, one result—what we built in 5.2
3. Real agents need loops: To handle multi-step tasks, verification, and error recovery (Chapters 14-17)
4. Self-correction is the key difference: Agents can observe results and adjust their approach

What we haven't covered (and won't until later chapters):

• How to compose execution pipelines (Chapter 6)
• How to handle structured output (Chapter 7)
• How to manage conversation memory (Chapter 8)
• How to define tools properly (Chapter 12)
• How to implement agent loops with self-correction (Chapters 14-17)
• How to make agents production-ready (Chapters 18-26)

This chapter was about the conceptual shift—understanding what makes a system exhibit agentic behavior. The implementation details come later.

In the next chapter, we'll continue building practical foundations: composing execution pipelines with LangChain Expression Language (LCEL). These are the building blocks you'll use when we implement real agents in Part IV.

The "aha!" moment is complete. You now understand that agentic systems are those where the LLM decides, and your code executes. Everything else—tools, loops, state management—is about making that decision-execution cycle more robust and capable.

Let's continue building the foundation.