5.1 LLM & Technical Fundamentals: Tokens, Context Windows & Sampling

LLM & Technical Fundamentals: Tokens, Context Windows & Sampling

Exam Blueprint Focus: LLM Fundamentals (5.2%) and Technical Fundamentals (6.1%) are the two largest sub-skills in the 16.8% Model Selection and Optimisation domain — together they outweigh model selection and cost management combined. They test the layer below the API: what a token is, what a context window actually holds, what the sampling parameters do, and how the HTTP and SDK surface is shaped.

Tokens: The Unit You Are Billed In

A token is a sub-word fragment produced by a byte-pair-encoding (BPE) tokenizer. English text averages roughly 4 characters, or 0.75 words, per token, but that ratio is an average and not a rule:

Two consequences engineers routinely get wrong:

1. You cannot estimate tokens from characters reliably, which is why /v1/messages/count_tokens exists. A 50,000-character English document and a 50,000-character JSON payload are not the same request.
2. Tokenizers change between model generations. Claude 4.7 and later models use a newer tokenizer that produces roughly 30% more tokens for the same text than the one used by Claude Sonnet 4.6 and earlier. A prompt that fit a budget on one generation may not on the next, even byte-for-byte unchanged.

Where tokens are counted

Every one of these consumes context and is billed:

• The system prompt, on every request.
• The full messages array — the API is stateless, so the entire history is re-sent and re-billed each turn.
• The tools array: names, descriptions, and JSON schemas.
• An automatic tool-use system prompt the API injects whenever tools is present: 354 tokens on Claude Sonnet 5 and 286 on Claude Opus 5 with tool_choice auto or none, rising to 474 and 406 with any or tool.
• thinking blocks, billed as output tokens even when the text is not returned to you.

Context Windows: Capacity, Not Budget

The context window is the maximum number of tokens a single request may contain — input plus the output it generates. Current capacities: 1,000,000 tokens on Claude Sonnet 5, Claude Opus 5, and Claude Fable 5.1; 200,000 on Claude Haiku 4.5. On the current tokenizer, 1M tokens is roughly 555,000 words.

Separately, max output caps the response: 128K tokens synchronously on Sonnet 5, Opus 5, and Fable 5.1, and 64K on Haiku 4.5. On the Message Batches API, several current models support up to 300K output tokens behind a beta header.

Three rules that follow:

• max_tokens is a ceiling, not a reservation. You are billed for tokens actually generated. Setting it generously costs nothing directly — but if the model hits it, generation stops mid-sentence with stop_reason: "max_tokens", which is a truncation bug, not a completion.
• Thinking and response text share the max_tokens ceiling. At high effort, a small max_tokens starves the answer.
• Capacity is not free. Every token in the window is re-billed on every turn it survives.

Sampling Parameters

After each forward pass the model produces a probability distribution over the vocabulary. Sampling parameters shape how the next token is drawn from it.

Practical guidance the exam rewards:

• Change one at a time. temperature and top_p both truncate the same distribution; tuning both together makes the effect uninterpretable.
• Low temperature is not determinism. It biases strongly toward the highest-probability path, but identical output across runs is not guaranteed by the API. Reproducibility comes from pinning the model snapshot and holding the prompt byte-identical — not from temperature: 0 alone.
• Structure beats sampling for structured output. If you need valid JSON, force a tool call with tool_choice rather than lowering temperature and hoping. Sampling shapes style; a schema enforces shape.
• Thinking constrains sampling. On manual extended-thinking models, temperature must be left at its default when thinking is enabled, and altering top_p/top_k restricts the reasoning exploration the mode exists to enable.

Technical Fundamentals: The HTTP and SDK Surface

Claude is a REST API over HTTPS. There is no WebSocket protocol and no session object on the server; streaming uses Server-Sent Events over ordinary HTTP, which matters for infrastructure design because SSE traverses standard HTTP proxies while WebSockets often do not.

POST https://api.anthropic.com/v1/messages
x-api-key: $ANTHROPIC_API_KEY
anthropic-version: 2023-06-01
content-type: application/json

Three headers, three facts:

• x-api-key, not Authorization: Bearer. This trips up developers arriving from other providers.
• anthropic-version is required and pins the API contract independently of the model. Version pinning and model pinning are two different decisions.
• anthropic-beta opts into pre-GA features; a request that needs a beta header and omits it fails rather than silently degrading.

The core endpoints:

What the SDKs add

The official SDKs (anthropic for Python, @anthropic-ai/sdk for TypeScript, plus Go, Java, C#, Ruby, and PHP) are thin wrappers over that REST surface, and they are worth using because of what they handle for you:

from anthropic import Anthropic

client = Anthropic()          # reads ANTHROPIC_API_KEY from the environment
resp = client.messages.create(
    model="claude-sonnet-5",
    max_tokens=1024,
    system="You are a precise technical assistant.",
    messages=[{"role": "user", "content": "Summarise this incident report."}],
)

• Automatic retries with exponential backoff on 429, 500, and 529 (2 retries by default), respecting retry-after.
• Streaming helpers that reassemble SSE deltas — including partial JSON in tool_use blocks — into whole objects.
• Typed response models, so a thinking block or a tool_use block is a typed object rather than a dict you must guess at.
• Credential handling from the environment, so the key never has to appear in source.

Deployment surfaces

The same Messages API is reachable through Amazon Bedrock, Google Cloud (Vertex AI), Microsoft Foundry, and Claude Platform on AWS. The model IDs differ per platform, and partner-operated platforms set their own lifecycle dates and their own pricing — a fact that matters when you are tracking retirement across a multi-cloud deployment.