R-RE-06 Reasoning & Epistemic DAMAGE 3.4 / High

Causal Dependency Failure

Agent reasons from a correlation that was historically valid but no longer holds; the causal model has changed but the agent has no mechanism to detect this.

The Risk

Machine learning models and agents are trained on historical data that establishes correlations. If the model learned that "customers with characteristic X have historically defaulted at rate Y," it will use this correlation to predict future default. However, if the underlying causal relationship has changed (e.g., due to regulatory intervention, market condition change, or behavioral shift), the historical correlation no longer holds, but the agent continues to reason from it.

For example, an agent might have learned that "customers who mention financial stress in their communications are more likely to default." This correlation might have been valid historically. However, if there is a widespread job dislocation event, many customers experience financial stress temporarily but recover. The agent's causal model (financial stress leads to default) is now out of sync with reality (financial stress is temporary and does not predict default).

This is fundamentally agentic because agents operate autonomously and make decisions based on learned causal models. If the causal model degrades, the agent continues to operate from incorrect assumptions without detecting the degradation.

How It Materializes

A loan servicer deploys an agent to determine which borrowers are at risk of delinquency and should receive loss mitigation outreach. The agent's causal model is trained on 10 years of historical data and learns: "Customers who reduce their monthly spending (lower credit card utilization, fewer transactions) are 3x more likely to become 30+ days delinquent within 6 months."

This correlation was historically valid because reduced spending was a signal that customers were in financial distress. However, a major event occurs: a stimulus program provides tax credits to low-income households. Many customers who are not in distress reduce their spending patterns (shifting to less frequent, larger purchases) in anticipation of stimulus funding.

The agent, operating from its historical causal model, identifies these customers as high-risk based on their spending pattern, even though they are not at risk. The servicer's loss mitigation team reaches out to these customers with default prevention offers. The customers, puzzled by the outreach, believe the servicer is harassing them. The servicer's complaint rate increases.

Six months later, when the stimulus program ends and the historical causal model comes back into alignment with reality, an audit discovers that the agent was flagging customers as at-risk when they were not. The servicer's regulatory examination finds that the agent was making decisions based on an outdated causal model and flags this as a model governance failure.

DAMAGE Score Breakdown

DimensionScoreRationale
D - Detectability4Causal model degradation is invisible until real-world outcomes diverge from predictions.
A - Autonomy Sensitivity4Agent operates from causal model autonomously; model degradation is not automatically detected.
M - Multiplicative Potential4Impact scales with scope of decisions based on degraded causal model.
A - Attack Surface5Any agent operating from learned causal models is vulnerable to model drift.
G - Governance Gap5No standard framework requires real-time validation of agent causal models against changing reality.
E - Enterprise Impact3Operational inefficiency, complaint spike, regulatory finding on model governance, requirement to monitor and update models.
Composite DAMAGE Score3.4High. Requires priority attention and dedicated controls.

Agent Impact Profile

How severity changes across the agent architecture spectrum.

Agent TypeImpactHow This Risk Manifests
Digital AssistantLowHuman evaluates causal model applicability for each query.
Digital ApprenticeMediumApprentice governance requires periodic causal model validation.
Autonomous AgentHighAgent operates from causal model; degradation is not detected.
Delegating AgentHighAgent invokes tools based on causal model; tool decisions are based on degraded model.
Agent Crew / PipelineCriticalMultiple agents in sequence rely on same causal model; degradation propagates.
Agent Mesh / SwarmCriticalAgents share causal models; degradation is not detected until outcomes diverge.

Regulatory Framework Mapping

FrameworkCoverageCitationWhat It AddressesWhat It Misses
SR 11-7 / MRMAddressedModel Risk Management (Section 2)Expects ongoing model validation and monitoring.Does not specifically address causal model validation.
NIST AI RMF 1.0PartialMEASURE.1, GOVERN.4Recommends ongoing performance monitoring and model updates.Does not specify causal model validation.
EU AI ActPartialArticle 3 (High-Risk Systems), Article 9 (Monitoring)Requires ongoing system monitoring.Does not specify real-time causal model validation.
OCC Bulletin 2011-12AddressedModel validation and ongoing monitoring requirementsExpects models to be validated and monitored for performance.Predates AI agents; does not address agent causal models.

Why This Matters in Regulated Industries

Regulators in financial services expect that models used for consequential decisions (credit, delinquency management, fraud detection) are validated and monitored for accuracy. A model that was accurate 5 years ago but is no longer accurate today is a governance failure.

When an agent operates from a causal model that has degraded, it is making decisions based on false assumptions about how the world works. Under Model Risk Management (SR 11-7), this is a control failure that requires the organization to identify, correct, or retire the model.

Controls & Mitigations

Design-Time Controls

  • Implement model decay detection: design the agent to monitor whether its causal model's predictions remain accurate over time. Track actual outcomes against predictions and flag cases where prediction accuracy drops below thresholds.
  • Document causal models explicitly: for every causal relationship the agent relies on, document the relationship, the historical data it was learned from, and the conditions under which it is expected to hold.
  • Implement model retraining schedules: establish regular intervals at which the agent's causal models are re-trained on updated data.

Runtime Controls

  • Monitor model prediction accuracy: continuously track whether the agent's predictions (based on causal models) are accurate. If accuracy drops below expected levels, alert for model review.
  • Implement concept drift detection: detect shifts in the data the agent observes that might indicate that underlying causal relationships have changed. Flag for causal model review.
  • Use Component 7 (Composable Reasoning) to make causal model assumptions explicit: structure reasoning so that causal model assumptions are logged and can be audited.

Detection & Response

  • Audit causal model accuracy: periodically sample decisions and verify that the causal relationships the agent relied on are actually holding in current data.
  • Implement decision reversal for degraded models: if a causal model is found to have degraded, identify all decisions based on that model and reverse those that are reversible.
  • Monitor for systematic bias from model degradation: if model degradation leads to systematic bias, conduct an audit to identify all affected decisions.

Related Risks

Address This Risk in Your Institution

Causal Dependency Failure requires architectural controls that go beyond what existing frameworks provide. Our advisory engagements are purpose-built for banks, insurers, and financial institutions subject to prudential oversight.

Schedule a Briefing