R-FM-04 Foundation Model & LLM DAMAGE 3.6 / High

Context Window Overflow and Information Loss

When inputs exceed the context window, content is truncated silently. Critical constraints defined early may be pushed out by subsequent content.

The Risk

Language models have context windows: maximum lengths of input they can process. Current models have context windows of 4K to 200K tokens. When an agent's combined prompt, instructions, retrieved documents, and conversation history exceed the context window size, some content must be truncated to fit. The model processes only the content that fits; content that does not fit is silently discarded.

This is dangerous when critical information is truncated. An agent is instructed to "follow these 47 compliance rules" and provided the rules as context. The agent also retrieves customer information, transaction data, and historical context. The total context exceeds the window size. The truncation algorithm typically prioritizes recent content and drops earlier content. The 47 compliance rules, provided at the beginning of the context, are truncated. The agent performs reasoning without the compliance rules.

The risk is amplified by silent failure: there is no error message, no warning that truncation occurred. The model processes whatever content fits and produces outputs as if the truncated content was never necessary. The institution has no visible indication that critical information was lost.

How It Materializes

A compliance officer uses an agent to review complex transaction scenarios and determine whether they are within institutional trading limits. The agent is given the full trading limit policy (40 KB, approximately 10,000 tokens), the current position data (30 KB, 7,000 tokens), the transaction details (10 KB, 2,500 tokens), and a conversation history asking for scenario analysis (5 KB, 1,250 tokens). Total: 85 KB, 20,750 tokens.

The agent's context window is 8K tokens. The model can only process 8K tokens. 12,750 tokens are truncated. The truncation algorithm drops content in reverse chronological order. The trading limit policy, provided first, is almost entirely truncated. The model receives: conversation history (complete, 1,250 tokens), transaction details (complete, 2,500 tokens), position data (partial, 3,500 tokens). The agent has no awareness of trading limits.

The agent analyzes the transaction and responds: "The transaction is consistent with the customer's investment profile and current positions. No issues identified." The agent has approved a transaction that violates trading limits because the trading limit policy was not in the context window.

The compliance officer reads the agent's recommendation and approves the transaction. The transaction is executed. Days later, a risk management audit discovers the transaction exceeded trading limits. The firm's risk controls failed due to a context window overflow. The truncation was silent; the agent produced a response without any indication that critical information was missing.

DAMAGE Score Breakdown

Dimension Score Rationale
D - Detectability 3 Truncation is silent; detection requires explicit logging of context window utilization or comparison of outputs to truncated vs. complete input.
A - Autonomy Sensitivity 2 Context overflow affects all autonomy levels; structural to LLM architecture.
M - Multiplicative Potential 3 Occurs when inputs are large; affects complex reasoning tasks more than simple tasks.
A - Attack Surface 2 Primarily structural issue. Adversary could intentionally provide large inputs to trigger truncation, but truncation occurs naturally.
G - Governance Gap 4 Risk governance assumes agents receive complete information for reasoning. Silent truncation breaks this assumption.
E - Enterprise Impact 2 Impact is typically limited to specific transaction or decision. Not systemic unless context overflow is frequent.
Composite DAMAGE Score 3.6 High. Requires priority attention and dedicated controls.

Agent Impact Profile

How severity changes across the agent architecture spectrum.

Agent Type Impact How This Risk Manifests
Digital Assistant Moderate Human may notice incomplete response or incoherent reasoning, suggesting truncation.
Digital Apprentice Moderate Agent may produce incomplete responses with less obvious indication that truncation occurred.
Autonomous Agent High Agent produces outputs without human review; truncation is invisible.
Delegating Agent Moderate Agent determines what context to provide to delegated models. May unknowingly truncate.
Agent Crew / Pipeline Moderate Multiple agents each have context window limits. Each agent may truncate.
Agent Mesh / Swarm Moderate Peer-to-peer agents each manage context windows independently.

Regulatory Framework Mapping

Framework Coverage Citation What It Addresses What It Misses
NIST AI RMF 1.0 Partial MAP 1.1, MAP 2.1 Recommends transparency and data quality monitoring. Does not address context window overflow.
EU AI Act Partial Article 24 (Documentation) Requires documentation of AI system properties. Does not address context window constraints.
MAS AIRG Minimal Section 6.1 (Governance) General governance requirements. Does not address context window overflow.
OWASP LLM Top 10 Partial LLM06 (Improper Output Filtering) Addresses output validation. Does not address input truncation.
BCBS 239 Minimal Data governance principles General data governance. Does not address context window overflow.

Why This Matters in Regulated Industries

In compliance and risk management, constraints and limits are critical. Trading limits, concentration limits, exposure limits, and compliance thresholds are constraints that must be applied to every decision. If constraints are truncated from context and not applied, the institution operates outside its risk framework. Regulators expect institutions to ensure constraints are enforced. An institution that allows constraints to be silently dropped from context violates risk governance principles.

In customer-facing contexts, missing critical information can produce inadequate or inappropriate outputs. If an agent is supposed to consider the customer's risk tolerance but risk tolerance data is truncated, the agent may provide inappropriate recommendations.

Controls & Mitigations

Design-Time Controls

  • For any agent performing consequential reasoning, establish a maximum useful context size. Ensure critical instructions, constraints, and rules fit within this size even with additional data.
  • Separate critical information from optional information: encode compliance rules, constraints, and limits in a compact, structured format.
  • Implement context prioritization: design prompts and context assembly to prioritize critical information (constraints, rules, limits) for early placement in context.
  • Require agents to explicitly state which context they are operating under. If context is truncated, agents should not proceed.

Runtime Controls

  • Monitor context window utilization: log the size of context provided to each agent request. Alert if context utilization approaches the model's window size (e.g., >80% utilized).
  • Implement context truncation detection: if context is truncated, log a warning and handle gracefully (decline to provide output, escalate to human review).
  • Use Component 4 (Blast Radius Calculator) to assess impact of truncation: if critical constraints are truncated, estimate how many decisions might be affected.
  • Use Component 10 (Kill Switch) to halt agents whose context windows are frequently full or overflowing.

Detection & Response

  • Conduct quarterly context window audits: for each agent, estimate typical context sizes, verify they fit within model window with margin for safety (e.g., use max 70% of window).
  • Monitor for outputs that violate known constraints: if agent output violates trading limits, compliance rules, or other constraints that should be in context, investigate whether truncation occurred.
  • Test agent robustness with truncated context: intentionally truncate constraints from context, verify agent handles gracefully.
  • Establish incident response for detected truncation: audit decisions made when truncation occurred, assess whether constraints were applied.

Related Risks

Address This Risk in Your Institution

Context Window Overflow and Information Loss requires architectural controls that go beyond what existing frameworks provide. Our advisory engagements are purpose-built for banks, insurers, and financial institutions subject to prudential oversight.

Schedule a Briefing