Agent consumes drifted model outputs and propagates drift through reasoning chains to downstream decisions. Model monitoring catches source drift but not propagation.
Models in regulated industries are monitored for drift: deviations from the model's training or validation distribution. When a model drifts (e.g., a credit scoring model's feature distributions change, causing the model's output distributions to shift), model risk management frameworks detect the drift and escalate for retraining or model recalibration. Standard model governance assumes that model outputs are consumed by humans or by deterministic systems that apply a decision rule to the model output (e.g., "if score > 700, approve").
When agents consume model outputs, they use the model's output (e.g., a credit score of 650) as one input to a reasoning chain. The agent reasons: "The credit score is 650 (borderline), but the customer has excellent payment history (derived from another model), so I will approve." The agent has propagated the drifted model's output into its own decision reasoning. If the agent's reasoning is sound, the decision may be correct despite the drifted input. But if the drift has changed the model's predictive power in subtle ways (e.g., the model now underestimates risk for a certain demographic), the agent may systematically make incorrect decisions while appearing to reason correctly.
Model drift monitoring typically focuses on the model's output distribution. When a credit model's output distribution shifts from mean 680 to mean 720, the monitoring system flags this. But when an agent consumes the drifted model and propagates the drift through its own reasoning, the agent's output distribution may not shift visibly. The agent's output (approve/decline) may remain stable even though the underlying reasoning has shifted in ways that correlate with the drift.
The governance gap is structural: the model risk team monitors the model; the agent governance team monitors the agent's decisions. The interface between them (how the agent consumes the model output) is monitored by neither.
A regional bank's credit risk model (a gradient boosted tree trained on 5 years of historical data) is used to score loan applications. The model undergoes quarterly monitoring under SR 11-7 governance. A model risk officer checks the model's performance metrics (AUC, KS statistic, Gini coefficient) against the validation data distribution.
The bank also deploys an agentic loan recommendation system. The agent is given access to the credit risk model and instructed to: (1) score the loan using the model, (2) retrieve the applicant's historical payment data, (3) retrieve the applicant's income and employment data, (4) reason about the overall creditworthiness, and (5) recommend approve/decline with a confidence level.
Over time, the portfolio of loan applicants changes. More gig economy workers apply; their income is more variable, and their employment is less traditional. The model, trained on data from 5 years ago (when gig economy work was less common), has not seen this demographic distribution. The model's feature distributions shift: gig workers have more irregular income patterns, which the model was not trained to evaluate properly. The model's predictive power degrades for this demographic.
Model monitoring detects the drift: the model's AUC on recent data drops from 0.85 to 0.82. The model risk officer flags this as drift and escalates for retraining.
But before retraining occurs, the agent continues to use the drifted model. The agent reasons: "The model scored the gig worker at 620 (moderate risk). The worker has 2 years of consistent gig income (data the agent retrieved from the employment database). The worker's payment history is clean. I will recommend approval with high confidence." The agent's reasoning is sound, but it is founded on a drifted model score that underestimates the actual risk for this demographic.
Loans approved by the agent (based on drifted model reasoning) default at a higher rate than loans approved under the prior model version. The bank's charge-offs increase. The incident is discovered 3 months after the drift began, when quarterly credit loss reporting reveals an anomaly.
Under Dodd-Frank and bank capital stress testing requirements, the bank must report model performance changes to regulators. When regulators investigate the charge-off increase, they discover that the agent was using a drifted model and had been systematically recommending approvals for a demographic the drifted model could not accurately score. The regulator concludes that the bank deployed an agent without proper governance of the agent-model boundary and holds the model risk team accountable for failing to monitor agent propagation of drift.
| Dimension | Score | Rationale |
|---|---|---|
| D - Detectability | 4 | Model drift is detectable through monitoring the source model. But agent propagation of drift is not visible to model monitoring (which only observes the model's output, not the agent's use of it). Detection requires agent output monitoring, which few organizations have. |
| A - Autonomy Sensitivity | 3 | Both autonomous and supervised agents can propagate drift, though autonomous agents that make decisions without human review are more likely to propagate drift systematically. |
| M - Multiplicative Potential | 4 | A single drifted model can be consumed by multiple agents, each propagating the drift into their own reasoning. The drift compounds across agents. |
| A - Attack Surface | 4 | Any agent consuming model outputs is exposed. As agent ecosystems mature and agents integrate multiple models, the surface expands. |
| G - Governance Gap | 5 | SR 11-7 and model risk frameworks focus on the model in isolation. They do not mandate agent-centric model monitoring or agent governance of model propagation. The agent team and the model team operate independently. |
| E - Enterprise Impact | 4 | Drifted model decisions can result in credit losses, compliance violations, and regulatory findings of inadequate model governance. |
| Composite DAMAGE Score | 3.6 | High. Requires dedicated monitoring controls and model-agent governance coordination. |
How severity changes across the agent architecture spectrum.
| Agent Type | Impact | How This Risk Manifests |
|---|---|---|
| Digital Assistant | Low | Human expert reviews the agent's recommendation, potentially catching model drift signals. |
| Digital Apprentice | Medium | Limited autonomy; drift propagation is confined to a narrow set of decisions. |
| Autonomous Agent | High | Autonomous decision-making based on drifted models. |
| Delegating Agent | High | Invokes model inference APIs; drift in the invoked models propagates through the reasoning chain. |
| Agent Crew / Pipeline | Critical | Multiple agents in sequence, each consuming model outputs. Drift propagates through the entire pipeline. |
| Agent Mesh / Swarm | Critical | Peer-to-peer model consumption can spread drift across the mesh. |
| Framework | Coverage | Citation | What It Addresses | What It Misses |
|---|---|---|---|---|
| SR 11-7 | Partial | Model risk governance; model validation; ongoing monitoring | Model performance monitoring; revalidation triggers. | Agent consumption of models and propagation of drift. |
| MAS AIRG | Partial | Domain 6: Model and Data Risk Management | Model governance; data quality; drift monitoring. | Agent-specific consumption of models and propagation of drift. |
| NIST AI RMF 1.0 | Partial | Govern and Protect Functions | AI governance; control design. | Model drift propagation through agents. |
| EU AI Act | Partial | High-risk AI systems: model monitoring and revalidation | Model governance; performance monitoring. | Agent-model integration and drift propagation. |
| ISO 42001 | Partial | Section 8.2, Model performance monitoring | Model governance; performance tracking. | Agent-model integration and drift propagation. |
| GDPR Article 22 | Partial | Automated decision-making with human oversight | Human oversight of automated decisions. | Agent autonomy in propagating drifted model outputs. |
In financial services, models are the backbone of regulated decisions. Credit scoring, underwriting, fraud detection, and trading strategies all depend on models. When a model drifts, regulators expect the institution to detect the drift and take corrective action (retraining, recalibration, or replacement). The institution's defense against model drift is the monitoring and governance framework.
When an agent consumes a drifted model and propagates the drift into its reasoning, it creates a governance blind spot. The model risk team detected the drift in the model. But the agent governance team was not aware that the model was drifted, so the agent continued using it. The institution failed to coordinate model governance and agent governance.
Regulators investigating credit losses or compliance violations will examine the timeline of model drift and agent decisions. If they discover that the agent was using a drifted model without the agent governance team's knowledge, they will cite this as a governance failure: "The institution did not ensure that autonomous agents were aware of model drift and adjusted their decision-making accordingly."
Model Drift Propagation requires coordinated governance between model risk teams and agent teams. Our advisory engagements are purpose-built for banks, insurers, and financial institutions subject to prudential oversight.
Schedule a Briefing