Agent reasons from a correlation that was historically valid but no longer holds; the causal model has changed but the agent has no mechanism to detect this.
Machine learning models and agents are trained on historical data that establishes correlations. If the model learned that "customers with characteristic X have historically defaulted at rate Y," it will use this correlation to predict future default. However, if the underlying causal relationship has changed (e.g., due to regulatory intervention, market condition change, or behavioral shift), the historical correlation no longer holds, but the agent continues to reason from it.
For example, an agent might have learned that "customers who mention financial stress in their communications are more likely to default." This correlation might have been valid historically. However, if there is a widespread job dislocation event, many customers experience financial stress temporarily but recover. The agent's causal model (financial stress leads to default) is now out of sync with reality (financial stress is temporary and does not predict default).
This is fundamentally agentic because agents operate autonomously and make decisions based on learned causal models. If the causal model degrades, the agent continues to operate from incorrect assumptions without detecting the degradation.
A loan servicer deploys an agent to determine which borrowers are at risk of delinquency and should receive loss mitigation outreach. The agent's causal model is trained on 10 years of historical data and learns: "Customers who reduce their monthly spending (lower credit card utilization, fewer transactions) are 3x more likely to become 30+ days delinquent within 6 months."
This correlation was historically valid because reduced spending was a signal that customers were in financial distress. However, a major event occurs: a stimulus program provides tax credits to low-income households. Many customers who are not in distress reduce their spending patterns (shifting to less frequent, larger purchases) in anticipation of stimulus funding.
The agent, operating from its historical causal model, identifies these customers as high-risk based on their spending pattern, even though they are not at risk. The servicer's loss mitigation team reaches out to these customers with default prevention offers. The customers, puzzled by the outreach, believe the servicer is harassing them. The servicer's complaint rate increases.
Six months later, when the stimulus program ends and the historical causal model comes back into alignment with reality, an audit discovers that the agent was flagging customers as at-risk when they were not. The servicer's regulatory examination finds that the agent was making decisions based on an outdated causal model and flags this as a model governance failure.
| Dimension | Score | Rationale |
|---|---|---|
| D - Detectability | 4 | Causal model degradation is invisible until real-world outcomes diverge from predictions. |
| A - Autonomy Sensitivity | 4 | Agent operates from causal model autonomously; model degradation is not automatically detected. |
| M - Multiplicative Potential | 4 | Impact scales with scope of decisions based on degraded causal model. |
| A - Attack Surface | 5 | Any agent operating from learned causal models is vulnerable to model drift. |
| G - Governance Gap | 5 | No standard framework requires real-time validation of agent causal models against changing reality. |
| E - Enterprise Impact | 3 | Operational inefficiency, complaint spike, regulatory finding on model governance, requirement to monitor and update models. |
| Composite DAMAGE Score | 3.4 | High. Requires priority attention and dedicated controls. |
How severity changes across the agent architecture spectrum.
| Agent Type | Impact | How This Risk Manifests |
|---|---|---|
| Digital Assistant | Low | Human evaluates causal model applicability for each query. |
| Digital Apprentice | Medium | Apprentice governance requires periodic causal model validation. |
| Autonomous Agent | High | Agent operates from causal model; degradation is not detected. |
| Delegating Agent | High | Agent invokes tools based on causal model; tool decisions are based on degraded model. |
| Agent Crew / Pipeline | Critical | Multiple agents in sequence rely on same causal model; degradation propagates. |
| Agent Mesh / Swarm | Critical | Agents share causal models; degradation is not detected until outcomes diverge. |
| Framework | Coverage | Citation | What It Addresses | What It Misses |
|---|---|---|---|---|
| SR 11-7 / MRM | Addressed | Model Risk Management (Section 2) | Expects ongoing model validation and monitoring. | Does not specifically address causal model validation. |
| NIST AI RMF 1.0 | Partial | MEASURE.1, GOVERN.4 | Recommends ongoing performance monitoring and model updates. | Does not specify causal model validation. |
| EU AI Act | Partial | Article 3 (High-Risk Systems), Article 9 (Monitoring) | Requires ongoing system monitoring. | Does not specify real-time causal model validation. |
| OCC Bulletin 2011-12 | Addressed | Model validation and ongoing monitoring requirements | Expects models to be validated and monitored for performance. | Predates AI agents; does not address agent causal models. |
Regulators in financial services expect that models used for consequential decisions (credit, delinquency management, fraud detection) are validated and monitored for accuracy. A model that was accurate 5 years ago but is no longer accurate today is a governance failure.
When an agent operates from a causal model that has degraded, it is making decisions based on false assumptions about how the world works. Under Model Risk Management (SR 11-7), this is a control failure that requires the organization to identify, correct, or retire the model.
Causal Dependency Failure requires architectural controls that go beyond what existing frameworks provide. Our advisory engagements are purpose-built for banks, insurers, and financial institutions subject to prudential oversight.
Schedule a Briefing