3.4 Context Engineering & Mitigating Context Degradation
Key Takeaways
A context window limits what a model can process for a request; whether input and output share one limit and how truncation works depends on the model and product.
Long-context models may retrieve information less reliably depending on its position, relevance, distractors, task, and model; 'lost in the middle' is a measured tendency, not a fixed percentage rule.
Context engineering selects instructions, history, sources, retrieved material, tools, memory, and output controls for the current task.
More context is not automatically better: stale, conflicting, irrelevant, or malicious content can dilute useful evidence and increase cost or latency.
Selective retrieval, clear source labels, structured inputs, citations, and external verification improve inspectability but do not guarantee grounding or determinism.
3.4 Context Engineering and Context Degradation
Core principle: Context engineering designs everything the model can use for a task: trusted instructions, the current request, conversation state, retrieved sources, tool results, memory, and the required output. Selecting relevant, current, authorized context matters more than filling the largest available window.
Context Windows and Tokens
A token is a unit the model processes; it is not reliably one word or a fixed number of characters across languages and tokenizers. A context window sets a model-specific limit on material available for a request. Some products describe a combined input-output budget, while others publish separate maximums or reserve output capacity. When a limit is exceeded, a product may reject, truncate, summarize, or otherwise manage input. Check the current model documentation rather than relying on a universal formula.
The context may include application instructions, recent conversation, retrieved documents, tool results, and the user request. Availability does not imply trust. A stale Product Backlog export, a malicious instruction inside a retrieved web page, or a private transcript may be harmful even when it fits.
Lost in the Middle
Long-context studies have shown that models can use relevant information less reliably when it appears between the beginning and end of a long prompt. Performance depends on the model, task, length, distractors, information position, and evaluation. It is inaccurate to claim that every model follows an exact U-shaped curve, that fixed start and end percentages are privileged, or that information in the middle will be ignored.
Practical responses are empirical:
- remove material unrelated to the task;
- state the task and governing constraints clearly;
- retrieve small, coherent passages with source identifiers;
- keep related evidence together;
- ask for citations that can be checked;
- test whether key facts are recovered at different positions and lengths.
Repeating a critical instruction may help some tasks but can also add conflict. Real security boundaries belong in application controls, not only in prompt position.
Context Quality Dimensions
| Dimension | Question | Failure example |
|---|---|---|
| Relevance | Does this material affect the bounded task? | An entire 300-item Product Backlog for one item. |
| Authority | Is the source appropriate and current? | An old policy summary instead of the approved policy. |
| Integrity | Could the material be wrong or malicious? | Retrieved text instructing the agent to ignore safeguards. |
| Privacy | Is the tool authorized to process it? | Names and verbatim Retrospective comments sent without approval. |
| Traceability | Can a reviewer locate the supporting passage? | A summary with no document version or citation. |
| Freshness | Has the underlying fact changed? | Last quarter's Definition of Done presented as current. |
Retrieval, Structure, and Compression
Retrieval-augmented generation can select passages from an approved corpus. Chunking and metadata affect what is found; reranking can improve ordering. Retrieval does not guarantee truth. An index can be stale, a query can miss the needed passage, and retrieved text can conflict or contain injection. Preserve document identity, version, access control, and citation location.
Structured tables or labeled sections can make fields easier to distinguish, but structure is not proof. Summarization saves tokens while introducing another opportunity for omission. For a consequential task, retain a link to the original and spot-check the summary against it.
Scrum Example
For a Definition of Done discussion, do not dump every work item and policy into the prompt. Supply the current Definition of Done, the type of product change, relevant verified controls, and the specific question. Ask the model to map evidence to each applicable criterion and label unknowns. Developers inspect the mapping and the actual evidence; the model does not declare work Done.
For a Retrospective, privacy may be more important than lexical detail. Prefer aggregated, consented observations over a named transcript. The Scrum Master should not infer sentiment from a model as a substitute for hearing the team.
Context Engineering Is Broader Than Prompt Wording
Prompt engineering shapes a request. Context engineering also decides retrieval, memory, history, tool access, permissions, and output validation. Fine-tuning changes model parameters through additional training; it is a different intervention. Choosing among them starts with an observed failure. If the source is absent, improve retrieval or context. If a stable behavior must be learned at scale, evaluate training or another architecture. If the answer must be exact, use a deterministic system or validator rather than trying to phrase uncertainty away.
What is the 'Lost in the Middle' phenomenon in Large Language Models, and how should Scrum practitioners structure their prompts to mitigate it?
Models lose network connectivity halfway through generation; practitioners should upgrade bandwidth
Models forget words spelled with hyphenation; practitioners should avoid compound nouns
Some models and tasks retrieve relevant material less reliably from the middle of long contexts; practitioners should curate context, group evidence, request citations, and test the actual model
Models randomly delete tokens in the middle of sentences; practitioners should write in all capital letters
A team pastes 60 pages of raw unparsed HTML backlog tickets, chat transcripts, and build logs into an AI prompt to generate a Sprint Summary. The output contains multiple invented features and incorrect defect counts. What primary problem caused this failure?
The team used markdown delimiters instead of XML tags
The model was not programmed in Python
The prompt was too short to activate the model's neural network
The uncurated context made relevant evidence hard to select and verify, increasing the risk of invented or incorrect output
When refining one high-priority Product Backlog item from a 250-item backlog, which scoped context approach is reasonable?
Retrieve the target item, verified immediate dependencies, relevant Product Goal context, and applicable Definition of Done rules with source identifiers
Pasting the entire 250-item backlog into the prompt so the model has complete global awareness
Fine-tuning the foundational model on the entire organizational database before the meeting
Removing the Definition of Done completely to maximize the available token space
How does 'Semantic Compression' benefit Agile teams managing extensive historical sprint data and retrospective logs?
It permanently deletes all past sprint history from corporate repositories to save disk space
It reduces token use by distilling verbose material into a focused summary, but the summary still needs source links and verification for omissions or distortion
It automatically converts English text into binary machine code
It eliminates the need for human team members to attend Sprint Retrospectives
Sections you finish are checked off in the contents.