2.1 Prompt Caching Mechanics & Breakpoints

2.1 Prompt Caching Mechanics & Breakpoints

Core Concept: Anthropic prompt caching reuses transformer prefix key-value (KV) tensors so a repeated prompt prefix is read back at 0.1x the base input price (a 90% discount) with a large reduction in time-to-first-token. Caching is configured with cache_control: {"type": "ephemeral"}, either automatically with one top-level field or explicitly on individual content blocks. Prompt caching is the single highest-leverage cost lever in the Cost and Token Management sub-skill.

---

The Prefix Caching Mechanism

When a large language model processes a sequence of input tokens, it computes Key and Value (KV) vector embeddings for each token at every transformer attention layer. In a conventional stateless API, every invocation forces the inference engine to recalculate these KV projections from token 0 through the end of the prompt—even if 95% of the prompt is identical to the preceding call. This redundant recomputation creates substantial GPU overhead, inflates time-to-first-token (TTFT) latency, and drives up token processing costs.

Anthropic's Prompt Caching introduces a stateful prefix caching layer across Anthropic's distributed inference infrastructure. When a prompt is submitted with designated caching instructions, the inference cluster calculates the KV representations for the designated prefix and saves them in an ephemeral, high-speed memory cache. On subsequent requests that share the exact same prefix sequence:

1. The inference cluster detects the prefix match starting from token 0.
2. It bypasses the transformer prefill computation for all cached tokens.
3. It streams the precomputed KV tensors directly into GPU memory.
4. The model begins generating completion tokens almost immediately, reducing TTFT by up to 85%.

Because serving pre-computed KV tensors takes far less work than running forward attention passes over thousands of tokens, Anthropic passes the efficiency through as a 0.1x multiplier on cache read tokens — a 90% discount off base input price. (Claude Fable 5.1 and Claude Mythos 5.1 go further, at 0.025x, or $0.25 per MTok.)

Standard Request (No Cache):
[System Prompt (3,000 tok)] + [Tools (1,500 tok)] + [User Query (50 tok)]
==> GPU recomputes attention for all 4,550 tokens from scratch every request.

Cached Request (Prefix Hit):
[System Prompt (3,000 tok)] + [Tools (1,500 tok)]  ==> READ FROM KV CACHE (0.1x cost, ~85% faster)
+ [User Query (50 tok)]                            ==> Standard computation (1.0x cost)

---

Two Ways to Enable Caching

Automatic caching (the recommended starting point)

Add a single cache_control field at the top level of the request. The system places the breakpoint on the last cacheable block and moves it forward automatically as the conversation grows:

{
  "model": "claude-sonnet-5",
  "max_tokens": 1024,
  "cache_control": { "type": "ephemeral" },
  "system": "You are an enterprise support intelligence agent...",
  "messages": [{ "role": "user", "content": "..." }]
}

This is the right default for a chat or agent loop, because it removes the most common source of cache misses: a hand-placed breakpoint that stops moving as history grows.

Explicit cache breakpoints (fine-grained control)

Place cache_control directly on individual content blocks when you need to control exactly what is cached — for example, to cache tool definitions and a document corpus under separate breakpoints so a tool change does not invalidate the corpus:

"cache_control": { "type": "ephemeral" }

"ephemeral" is the only supported cache type.

Placement Rules and Limits

1. Maximum Breakpoints: A request may carry at most 4 cache breakpoints. Automatic caching consumes one of those 4 slots; if 4 explicit breakpoints already exist, the request returns a 400 error.
2. Supported Structures: Breakpoints can be declared inside:
   • System Prompts: Within individual content blocks of the system parameter array.
   • Tool Definitions: Directly inside individual tool schemas within the tools parameter array.
   • Message History: On content blocks within user or assistant messages in the messages array.
3. Cumulative Prefix Evaluation: A cache breakpoint marks the boundary of a cached prefix. Everything from token 0 up to and including the block carrying the cache_control marker forms one cached checkpoint. The prefix is built in a fixed order: tools -> system -> messages, so a change to tools invalidates everything after it.
4. The 20-position lookback. If the hash at your breakpoint does not match, the system walks backward one block at a time for up to 20 positions, checking whether an earlier prefix hash matches a previously cached entry. Consecutive tool_use blocks count as one position, as do consecutive tool_result blocks. This is why a well-placed breakpoint still often hits after a small edit — and why an agent that appends more than 20 blocks between requests can fall off the lookback window entirely.

Python SDK Implementation Example

The following code illustrates a production setup placing cache breakpoints across system instructions, tool definitions, and conversation turns:

import anthropic

client = anthropic.Anthropic()

response = client.messages.create(
    model="claude-sonnet-5",
    max_tokens=1024,
    # 1. Cache Breakpoint on Static System Prompt
    system=[
        {
            "type": "text",
            "text": "You are an enterprise support intelligence agent. Here is the operational compliance handbook...\n[3,000 tokens of guidelines]",
            "cache_control": {"type": "ephemeral"}  # Breakpoint 1
        }
    ],
    # 2. Cache Breakpoint on Tools Library
    tools=[
        {
            "name": "lookup_customer_record",
            "description": "Retrieves comprehensive CRM record by customer ID.",
            "input_schema": {
                "type": "object",
                "properties": {"customer_id": {"type": "string"}},
                "required": ["customer_id"]
            }
        },
        {
            "name": "execute_refund",
            "description": "Issues refund transaction against payment gateway.",
            "input_schema": {
                "type": "object",
                "properties": {
                    "transaction_id": {"type": "string"},
                    "amount_cents": {"type": "integer"}
                },
                "required": ["transaction_id", "amount_cents"]
            },
            "cache_control": {"type": "ephemeral"}  # Breakpoint 2: caches all tools up to here
        }
    ],
    # 3. Cache Breakpoint on Historical Multi-Turn Conversation
    messages=[
        {
            "role": "user",
            "content": "Here is the customer diagnostic dump from server logs: [1,200 tokens of log lines]"
        },
        {
            "role": "assistant",
            "content": [
                {
                    "type": "text",
                    "text": "I have parsed the diagnostic logs and identified three memory pressure warnings.",
                    "cache_control": {"type": "ephemeral"}  # Breakpoint 3: caches log context + assistant turn
                }
            ]
        },
        {
            "role": "user",
            "content": "What remediation actions should I execute first?"  # Dynamic tail (not cached)
        }
    ]
)

---

Token Thresholds and Silent Bypass Mechanics

To ensure GPU memory allocations remain efficient, Anthropic enforces minimum token thresholds for prompt caching. The caching engine will only write a prefix to the KV cache if the cumulative number of tokens from token 0 up to the breakpoint meets or exceeds the model's threshold.

Note the shape of that table: the thresholds do not track the tier hierarchy. Claude Haiku 4.5 needs an 8x larger prefix than Claude Opus 5 before anything is cached at all. A 2,000-token system prompt caches on Opus 5 and Sonnet 5 and silently does not cache on Haiku 4.5 — which is exactly the trap on a cost-optimisation question that routes cheap traffic to Haiku.

The Silent Bypass Rule

A frequent trap on the certification exam involves sub-threshold behavior. If a developer includes cache_control: {"type": "ephemeral"} on a prompt block where the cumulative prefix contains fewer tokens than the threshold (for example, 650 tokens on Claude Sonnet 5, or 3,000 tokens on Claude Haiku 4.5):

• The API does not reject the request.
• No HTTP 400 error or validation exception is thrown.
• The request silently bypasses caching: no cache write occurs, no cache read occurs, and all input tokens are billed at standard base input rates.
• In the response telemetry, cache_creation_input_tokens and cache_read_input_tokens will both register as 0.

---

Cache Lifetime, Eviction, and Economic Modeling

Prompt cache entries have a default Time-to-Live (TTL) of 5 minutes, with an opt-in 1-hour tier:

{ "cache_control": { "type": "ephemeral", "ttl": "1h" } }

The TTL is measured from the start of the request that writes or reads the cache, not from when the response finishes — so a long generation spends part of its own cache lifetime.

The Sliding TTL Window

The 5-minute TTL is not an absolute, immutable timer; it operates as a sliding refresh window:

• When a prompt is first cached, the system writes the entry and sets the 5-minute expiration countdown.
• Whenever a subsequent request matches the cached prefix and triggers a cache read, the 5-minute TTL automatically resets back to 5 minutes.
• As long as your application hits the cache at least once every 4 minutes and 59 seconds, the cached prefix can remain active indefinitely in memory.
• If 5 minutes pass without any request hitting the prefix, the entry is evicted. The next request with that prefix must perform a fresh cache write.

Token Pricing Economics

Caching introduces three input-token billing tiers, expressed as multipliers on the model's base input price:

Break-Even Analysis

The multipliers make break-even arithmetic simple and exam-testable:

A 5-minute cache write costs 1.25x and a read costs 0.1x, so caching pays off after a single read. A 1-hour write costs 2x, so it pays off after two reads.

Worked example on Claude Sonnet 5 ($2.00/MTok base input):

• Cache write (5m): $2.50 per MTok ($2.00 x 1.25)
• Cache read: $0.20 per MTok ($2.00 x 0.10)

For a 4,000-token stable prefix:

• Without caching, 2 calls: 2 x 4,000 x $2.00/M = $0.0160
• With caching, 2 calls (1 write + 1 read): (4,000 x $2.50/M) + (4,000 x $0.20/M) = $0.0100 + $0.0008 = $0.0108 — a 32.5% saving after one read.
• Over 20 calls in a session:
  • Un-cached: 20 x 4,000 x $2.00/M = $0.160
  • Cached: (4,000 x $2.50/M) + 19 x (4,000 x $0.20/M) = $0.0100 + $0.0152 = $0.0252 — an 84.25% reduction.

Note that the percentages are identical to the Opus 5 or Haiku 4.5 versions of the same calculation: because every tier uses the same 1.25x / 0.1x multipliers, the savings ratio is model-independent and only the absolute dollars move.

Comprehensive Cost Reference Table

These multipliers stack with other pricing modifiers, including the 50% Batch API discount.

---

Telemetry: Inspecting the usage Object

Production observability requires monitoring how effectively your application hits the prompt cache. Every response from the Messages API returns a usage object containing granular token attribution:

{
  "id": "msg_01ABcDeF...",
  "type": "message",
  "role": "assistant",
  "content": [{"type": "text", "text": "Customer account balance is verified."}],
  "model": "claude-sonnet-5",
  "stop_reason": "end_turn",
  "usage": {
    "input_tokens": 42,
    "cache_creation_input_tokens": 2850,
    "cache_read_input_tokens": 0,
    "output_tokens": 68
  }
}

Understanding Telemetry Fields

• cache_creation_input_tokens: The count of tokens written to the cache during this request. Billed at 1.25x base rate.
• cache_read_input_tokens: The count of tokens successfully read from the existing cache. Billed at 0.10x base rate.
• input_tokens: The un-cached delta tokens. These are tokens appearing after the last cache breakpoint, or tokens that fell below minimum thresholds. Billed at standard 1.0x base rate.
• output_tokens: Completion tokens generated by the model. Billed at standard output rates.

On a successful second request hitting the cache:

"usage": {
  "input_tokens": 42,
  "cache_creation_input_tokens": 0,
  "cache_read_input_tokens": 2850,
  "output_tokens": 74
}

---

Architectural Best Practices: Prefix Stability & Invalidation Traps

The single most critical architectural principle for prompt caching is Prefix Stability. The caching engine performs strict, deterministic prefix matching starting from token 0. If even one character, whitespace, or token changes early in the prompt, the cache lookup fails for that point and invalidates all subsequent breakpoints.

1. Optimal Content Ordering Hierarchy

To maximize cache hit rates, structure prompts strictly from most static to most dynamic:

[Token 0] ──────────────────────────────────────────────────────────> [End of Prompt]
┌────────────────────────┬───────────────────┬──────────────────────┬────────────────┐
│  Static System Prompt  │  Tool Definitions │ Long Reference Docs  │ Dynamic User   │
│  & Base Instructions   │  (Schemas)        │ & Few-Shot Examples  │ Query & Turns  │
│  [Breakpoint 1]        │  [Breakpoint 2]   │ [Breakpoint 3]       │ [Breakpoint 4] │
└────────────────────────┴───────────────────┴──────────────────────┴────────────────┘
         STATIC ──────────────────────────────────────────────> DYNAMIC

2. Common Cache Invalidation Anti-Patterns

Developers frequently inadvertently destroy caching benefits through these common traps:

• Dynamic Timestamps in System Prompts: Placing "Current Date/Time: 2026-09-10 05:33:38" at the top of a system prompt alters token 0-15 on every request. This ensures every request produces a 100% cache miss, forcing a continuous stream of expensive 1.25x cache creation writes! Dynamic timestamps must be placed at the very end of the prompt (in the dynamic user message) or after all static cached blocks.
• Randomized or User-Specific Identifiers: Prepending session IDs, user IDs, or tracing headers in the system prompt fragments the cache per user rather than sharing a single global cache across all users.
• Changing tool definitions at all: because the prefix order is tools -> system -> messages, editing, adding, or removing a tool invalidates the entire cache, including system and message breakpoints. Toggling web search or citations invalidates system and message caches. Changing tool_choice, adding or removing images, or changing the thinking or effort configuration invalidates the message blocks.
• Unordered Tool Schemas: In languages like Python or Go where dictionary iteration order can vary if not sorted, serializing tool lists non-deterministically causes tools to appear in random order, breaking prefix token sequences. Always sort tool definitions deterministically before passing them to the API.
• Variable JSON Formatting: Subtle changes in formatting (such as changing indentation from 2 spaces to 4 spaces, or altering key order in few-shot JSON examples) changes the token sequence and triggers cache eviction.

3. Multi-Turn Conversation Breakpoint Strategy

In multi-turn chat interactions, each turn appends new tokens. If you place a breakpoint only on the system prompt, you benefit from caching the system prompt, but each turn recomputes all prior dialogue history. The recommended pattern is to maintain a sliding breakpoint on the penultimate turn:

• Keep Breakpoint 1 on the static system prompt.
• Keep Breakpoint 2 on the tool declarations.
• Place Breakpoint 3 on the last assistant response before the user's latest query. This ensures that the entire dialogue history up to the current turn is read from cache at a 90% discount, while only the user's latest message and Claude's new reply are processed at standard rates.