7.2 Workflow Patterns: Prompt Chaining & Routing

Workflow Patterns: Prompt Chaining & Routing

Exam Blueprint Focus: The CCDV-F examination places heavy emphasis on building reliable, cost-effective deterministic workflows. Candidates must master Prompt Chaining with programmatic validation gates (fail-fast checks, circuit breaking, and selective retries) as well as Intelligent Routing (dispatching queries across model tiers like Claude Haiku 4.5 and Claude Sonnet 5, and partitioning monolithic prompts into specialized domain handlers).

---

The Prompt Chaining Pattern: Sequential Task Decomposition

In production LLM application design, attempting to solve a multi-faceted business problem using a single massive prompt is a well-known anti-pattern. Monolithic prompts that ask the model to ingest guidelines, extract structured data, perform complex reasoning, cross-reference rules, draft prose, and format outputs frequently suffer from instruction drift, missed edge cases, and high failure rates.

Prompt Chaining solves this by decomposing a complex, multi-stage task into a sequential pipeline of smaller, highly targeted prompts where the output of step $N$ becomes part of the input context for step $N+1$.

[Raw User Input]
       │
       ▼
┌──────────────┐      Structured     ┌──────────────┐      Refined Output    ┌──────────────┐
│ Step 1: LLM  │ ──────────────────► │ Step 2: LLM  │ ─────────────────────► │ Step 3: LLM  │ ──► [Final Response]
│ (Extraction) │    Intermediate     │ (Reasoning)  │       Intermediate     │ (Formatting) │
└──────────────┘       Payload       └──────────────┘         Payload        └──────────────┘

Why Prompt Chaining Outperforms Monolithic Prompts

1. Cognitive Load Reduction: Large language models have finite attentional capacity. By constraining each call to a single cognitive task (e.g., Step 1: extract data; Step 2: verify against policy; Step 3: draft email), the model achieves significantly higher accuracy and strict adherence to negative constraints.
2. Context Window Hygiene: Rather than accumulating the entire conversational reasoning trace, intermediate steps can discard irrelevant background text and pass only clean, distilled artifacts to subsequent steps.
3. Modular Unit Testing and Evals: Each step in the chain has explicit inputs and outputs, allowing engineering teams to write isolated unit tests, track token costs per stage, and run regression evaluations against specific prompt versions without testing the entire system end-to-end.
4. Targeted Model Selection: Different steps can utilize different models. Step 1 (simple extraction) can run on Claude Haiku 4.5 ($1.00/MTok), Step 2 (deep synthesis) on Claude Sonnet 5 ($2.00/MTok), and Step 3 (formatting) back on Haiku, slashing overall pipeline costs.

---

Programmatic Gates: Deterministic Verification Between Steps

A naive prompt chain simply pipes the raw string output of one LLM directly into the input of the next. In an enterprise system, this is fragile: if Step 1 hallucinates or outputs malformed text, that error cascades down the pipeline, causing subsequent steps to fail.

An enterprise-grade prompt chain introduces Programmatic Gates (deterministic code validations) between LLM steps:

[Step 1: LLM Extraction]
           │
           ▼
┌──────────────────────────────────────┐
│ Programmatic Gate (Host Code)        │
│ - Pydantic / Zod Schema Validation   │
│ - Deterministic Business Rules       │
│ - Regex & Range Assertions           │
└──────────────────────────────────────┘
      │                        │
   [Pass]                   [Fail]
      │                        ▼
      ▼                 ┌──────────────────────────────────────┐
[Step 2: LLM Synthesis] │ Circuit Breaker / Targeted Retry     │
                        │ - Retry Step 1 with validation error │
                        │ - Or fail-fast to human escalation   │
                        └──────────────────────────────────────┘

Core Functions of Programmatic Gates

1. Early Error Termination (Fail-Fast)

If the output of Step 1 violates fundamental invariants (e.g., missing required JSON fields, negative monetary values, invalid account IDs), execution halts immediately. This prevents burning tokens and latency on downstream steps that are guaranteed to fail.

2. Circuit Breaking

If a step fails validation repeatedly (e.g., three consecutive attempts), the circuit breaker trips. The system aborts the automated pipeline and cleanly falls back to a default safe response, an asynchronous human-in-the-loop queue, or an alert to the operations team.

3. Targeted Retries with Error Feedback

When an intermediate step fails validation, the system does not need to restart the entire multi-step pipeline from scratch. Instead, it re-invokes only that specific step, injecting the deterministic error message back into the prompt:

"Your previous output failed validation with error: 'Field interest_rate must be a float between 0.0 and 1.0, received 4.5'. Please correct this and re-generate the JSON object."

---

The Routing Pattern: Intent Classification & Dynamic Dispatch

While Prompt Chaining connects steps sequentially, Routing is a branching architecture. The routing pattern classifies an incoming request once and dispatches it to a specialized downstream handler, model tier, or system prompt.

                               [Incoming User Request]
                                          │
                                          ▼
                         ┌──────────────────────────────────┐
                         │      Classifier / Router         │
                         │   (Claude Haiku 4.5 / Schema)    │
                         └──────────────────────────────────┘
                                    │   │   │
                ┌───────────────────┘   │   └───────────────────┐
                ▼                       ▼                       ▼
    [Route A: Technical]       [Route B: Billing]      [Route C: General/Fallback]
    - Claude Sonnet 5        - Claude Haiku 4.5      - Claude Haiku 4.5
    - Deep Coding Prompt       - Account DB Schema     - FAQ Knowledge Base
    - Code Sandbox Tools       - SQL Query Tools       - Standard Persona

The Two Primary Dimensions of Routing

1. Complexity-Based Model Routing (Cost & Latency Optimization)

In production environments, user queries follow a power-law distribution: 60% to 80% of queries are routine (e.g., checking status, FAQ lookup, basic formatting), while only 20% to 40% require frontier reasoning.

• Sending every query to Claude Sonnet 5 ($2.00 / $10.00 per MTok) wastes massive compute budgets.
• Sending complex queries to Claude Haiku 4.5 ($1.00 / $5.00 per MTok) leads to customer dissatisfaction and task failure.
• The Router Solution: A lightweight Claude Haiku 4.5 classification call evaluates the complexity of the request in under 300ms. Simple queries are handled directly by Haiku, while complex reasoning queries are routed to Sonnet. This architecture reduces overall blended operating costs by 60% to 75% while maintaining top-tier quality.

2. Task-Based Prompt Specialization (Eliminating Monolithic System Prompts)

A common anti-pattern is writing a single, sprawling 5,000-token system prompt that instructs Claude how to handle billing questions, security disputes, technical bug reports, API integration, and general chit-chat.

• The Problem: Monolithic prompts suffer from token waste (every user turn pays for 5,000 tokens of mostly irrelevant instructions) and degraded instruction adherence.
• The Router Solution: The router classifies the user's intent into a specific domain ("billing", "technical_support", "security", "sales"). The request is then dispatched to a dedicated micro-prompt of only 400 tokens tailored specifically to that domain.

---

Implementation Patterns in Python

1. Robust Classification via Anthropic Tool Choice

When implementing a router, relying on free-form text output (e.g., asking the model to respond with just the word "BILLING") is brittle. The model might output "The category is BILLING" or "\nBILLING.", breaking application code.

The production best practice is to define a classification tool with a strict enum schema and force its selection using tool_choice:

import anthropic
from typing import Literal

client = anthropic.Anthropic()

# Define the routing schema using Anthropic tool definitions
routing_tool = {
    "name": "route_inquiry",
    "description": "Classify the user inquiry into the appropriate operational category.",
    "input_schema": {
        "type": "object",
        "properties": {
            "category": {
                "type": "string",
                "enum": ["billing_refund", "technical_code", "general_faq", "account_security"],
                "description": "The classified operational category."
            },
            "complexity": {
                "type": "string",
                "enum": ["low", "high"],
                "description": "Assess whether deep multi-step reasoning or code analysis is required."
            },
            "confidence": {
                "type": "number",
                "description": "Confidence score between 0.0 and 1.0."
            }
        },
        "required": ["category", "complexity", "confidence"]
    }
}

def route_incoming_query(user_query: str) -> dict:
    # Use Claude Haiku 4.5 for fast, low-cost classification
    response = client.messages.create(
        model="claude-haiku-4-5-20251001",
        max_tokens=150,
        temperature=0.0,
        tools=[routing_tool],
        tool_choice={"type": "tool", "name": "route_inquiry"},
        messages=[{"role": "user", "content": user_query}]
    )

    # Extract deterministic structured arguments directly from tool_use block
    for block in response.content:
        if block.type == "tool_use" and block.name == "route_inquiry":
            return block.input

    # Safe fallback for unexpected edge cases
    return {"category": "general_faq", "complexity": "low", "confidence": 0.5}

2. End-to-End Chaining with Pydantic Validation Gate

Here is a complete, production-grade pattern combining sequential chaining with a programmatic validation gate:

from pydantic import BaseModel, Field, ValidationError

class ExtractedFinancialData(BaseModel):
    company_name: str
    fiscal_year: int = Field(ge=2000, le=2030)
    revenue_usd: float = Field(gt=0)
    net_income_usd: float

def step_1_extract_metrics(raw_filing_text: str) -> str:
    """Step 1: Extract financial metrics into raw JSON."""
    response = client.messages.create(
        model="claude-haiku-4-5-20251001",
        max_tokens=500,
        system="Extract company financial metrics into JSON with keys: company_name, fiscal_year, revenue_usd, net_income_usd.",
        messages=[
            {"role": "user", "content": raw_filing_text},
            {"role": "assistant", "content": "{"}  # Prefill to force JSON
        ]
    )
    return "{" + response.content[0].text

def programmatic_gate(raw_json_str: str) -> ExtractedFinancialData:
    """Programmatic Gate: Validate schema and business rules."""
    try:
        data = ExtractedFinancialData.model_validate_json(raw_json_str)
        # Enforce business logic constraint
        if data.net_income_usd > data.revenue_usd:
            raise ValueError("Net income cannot exceed total revenue.")
        return data
    except (ValidationError, ValueError) as e:
        raise ValueError(f"Programmatic Gate Failed: {e}")

def step_2_generate_briefing(validated_data: ExtractedFinancialData) -> str:
    """Step 2: Generate executive briefing on validated data."""
    prompt = (
        f"Generate an executive briefing for {validated_data.company_name} (FY{validated_data.fiscal_year}).\n"
        f"Revenue: ${validated_data.revenue_usd:,.2f} | Net Income: ${validated_data.net_income_usd:,.2f}."
    )
    response = client.messages.create(
        model="claude-sonnet-5",
        max_tokens=600,
        system="You are a senior equity analyst. Provide a 3-bullet financial briefing.",
        messages=[{"role": "user", "content": prompt}]
    )
    return response.content[0].text

def execute_pipeline(raw_filing: str) -> str:
    # Step 1: LLM Extraction
    raw_json = step_1_extract_metrics(raw_filing)
    
    # Programmatic Gate: Deterministic Validation & Circuit Breaker
    try:
        validated_metrics = programmatic_gate(raw_json)
    except ValueError as err:
        # Circuit Breaker: Fail-fast or route to human auditor
        return f"Pipeline Aborted by Gate: {err}"
    
    # Step 2: High-Reasoning LLM Briefing
    return step_2_generate_briefing(validated_metrics)

---

Comparing Prompt Chaining vs. Routing