3.4 Context Engineering & Mitigating Context Degradation

3.4 Context Engineering and Context Degradation

Core principle: Context engineering designs everything the model can use for a task: trusted instructions, the current request, conversation state, retrieved sources, tool results, memory, and the required output. Selecting relevant, current, authorized context matters more than filling the largest available window.

---

Context Windows and Tokens

A token is a unit the model processes; it is not reliably one word or a fixed number of characters across languages and tokenizers. A context window sets a model-specific limit on material available for a request. Some products describe a combined input-output budget, while others publish separate maximums or reserve output capacity. When a limit is exceeded, a product may reject, truncate, summarize, or otherwise manage input. Check the current model documentation rather than relying on a universal formula.

The context may include application instructions, recent conversation, retrieved documents, tool results, and the user request. Availability does not imply trust. A stale Product Backlog export, a malicious instruction inside a retrieved web page, or a private transcript may be harmful even when it fits.

Lost in the Middle

Long-context studies have shown that models can use relevant information less reliably when it appears between the beginning and end of a long prompt. Performance depends on the model, task, length, distractors, information position, and evaluation. It is inaccurate to claim that every model follows an exact U-shaped curve, that fixed start and end percentages are privileged, or that information in the middle will be ignored.

Practical responses are empirical:

• remove material unrelated to the task;
• state the task and governing constraints clearly;
• retrieve small, coherent passages with source identifiers;
• keep related evidence together;
• ask for citations that can be checked;
• test whether key facts are recovered at different positions and lengths.

Repeating a critical instruction may help some tasks but can also add conflict. Real security boundaries belong in application controls, not only in prompt position.

Context Quality Dimensions

Retrieval, Structure, and Compression

Retrieval-augmented generation can select passages from an approved corpus. Chunking and metadata affect what is found; reranking can improve ordering. Retrieval does not guarantee truth. An index can be stale, a query can miss the needed passage, and retrieved text can conflict or contain injection. Preserve document identity, version, access control, and citation location.

Structured tables or labeled sections can make fields easier to distinguish, but structure is not proof. Summarization saves tokens while introducing another opportunity for omission. For a consequential task, retain a link to the original and spot-check the summary against it.

Scrum Example

For a Definition of Done discussion, do not dump every work item and policy into the prompt. Supply the current Definition of Done, the type of product change, relevant verified controls, and the specific question. Ask the model to map evidence to each applicable criterion and label unknowns. Developers inspect the mapping and the actual evidence; the model does not declare work Done.

For a Retrospective, privacy may be more important than lexical detail. Prefer aggregated, consented observations over a named transcript. The Scrum Master should not infer sentiment from a model as a substitute for hearing the team.

Context Engineering Is Broader Than Prompt Wording

Prompt engineering shapes a request. Context engineering also decides retrieval, memory, history, tool access, permissions, and output validation. Fine-tuning changes model parameters through additional training; it is a different intervention. Choosing among them starts with an observed failure. If the source is absent, improve retrieval or context. If a stable behavior must be learned at scale, evaluate training or another architecture. If the answer must be exact, use a deterministic system or validator rather than trying to phrase uncertainty away.