Organization applies model risk management framework (SR 11-7) to agents without recognizing that agentic risks are categorically different from model risks.
The Federal Reserve issued SR 11-7, a guidance on model risk management. The guidance applies to all financial services models used for decision-making, including AI and machine learning models. It requires that models be validated, monitored for performance degradation, and subject to governance oversight.
Many organizations have adapted SR 11-7 to cover AI/ML models. The framework is well-understood and has established governance processes. When agentic systems are deployed, organizations often apply the same SR 11-7 framework, treating agents as "models."
However, agentic systems are categorically different from traditional models. A traditional model has fixed architecture and behavior. An agent learns, adapts, coordinates with other agents, and makes autonomous decisions. SR 11-7 does not account for these agentic characteristics.
Model risk conflation occurs when organizations apply SR 11-7 to agents without recognizing the categorical differences. The organization believes it is governing agents adequately because it is applying a well-established model risk framework. But the framework does not address agentic-specific risks.
A bank applies SR 11-7 to its agentic trading system. Under SR 11-7, the bank validates the agent through pre-deployment testing on historical data, monitors real-world performance against test-set performance, establishes governance oversight with a trading manager reviewing major trades, and implements escalation procedures if the agent's performance degrades.
The bank believes it is adequately governing the agent because it is applying SR 11-7. However, SR 11-7 was designed for models, not agents. It does not address online learning (the agent learns from trading feedback and its behavior changes at runtime), autonomy governance (the agent makes trading decisions autonomously without specific per-decision human review), emergent behaviors (the agent coordinates with risk management and compliance systems), or tool use (the agent invokes APIs to execute trades).
Six months after deployment, the agent begins to exhibit unexpected behavior. The agent's learning has led to a change in its trading strategy. The agent is now making more aggressive trades than initially. The Sharpe ratio is higher, but the maximum drawdown is also higher. The agent is taking more risk than the initial performance indicated.
The bank's performance monitoring detects the Sharpe ratio improvement and does not flag this as a problem. The bank believes the agent is performing better. But the agent has developed a riskier strategy than was anticipated. If the market becomes more volatile, the agent's aggressive strategy could lead to large losses. The bank did not recognize that the agent's learning had changed its risk profile, because SR 11-7 does not contemplate model learning. Regulators examining the bank's governance note that SR 11-7 alone is insufficient for agents.
| Dimension | Score | Rationale |
|---|---|---|
| D - Detectability | 4 | Model risk conflation is not immediately visible. The organization believes it is adequately governing because it is applying an established framework. The conflation becomes apparent when agentic-specific risks materialize. |
| A - Autonomy Sensitivity | 4 | Model risk conflation affects autonomous agents most severely. Agents with human oversight are less vulnerable because humans may recognize risks that SR 11-7 does not address. |
| M - Multiplicative Potential | 3 | Model risk conflation affects organizations that apply SR 11-7 to agents without recognizing the categorical differences. |
| A - Attack Surface | 3 | Model risk conflation is not a direct security vulnerability. It is a governance and risk management issue. |
| G - Governance Gap | 4 | Most financial services organizations apply SR 11-7 to all models and agents. Few recognize that agentic systems require governance frameworks beyond SR 11-7. |
| E - Enterprise Impact | 4 | Model risk conflation can lead to inadequate governance of agentic systems. Impact becomes apparent when agentic-specific risks materialize. |
| Composite DAMAGE Score | 3.3 | High. Requires dedicated controls and regular monitoring. |
How severity changes across the agent architecture spectrum.
| Agent Type | Impact | How This Risk Manifests |
|---|---|---|
| Digital Assistant | Low | DA operates with human oversight. Humans can recognize agentic-specific risks. Model risk conflation is less problematic because humans provide additional governance. |
| Digital Apprentice | Low | AP is supervised. Supervisors can recognize agentic-specific risks. Supervision provides governance beyond SR 11-7. |
| Autonomous Agent | High | AA operates independently. If governance relies only on SR 11-7 without agentic-specific controls, agentic-specific risks are not addressed. |
| Delegating Agent | Medium | DL invokes tools. SR 11-7 does not address tool governance. Tool use governance is not implemented, leading to risk. |
| Agent Crew / Pipeline | High | CR chains agents. SR 11-7 does not address agent coordination. Coordination risks are not governed. |
| Agent Mesh / Swarm | High | MS features emergent coordination. SR 11-7 does not address emergence. Emergent risks are not governed. |
| Framework | Coverage | Citation | What It Addresses | What It Misses |
|---|---|---|---|---|
| SR 11-7 | Partial | Model risk management for all models including AI/ML | Model risk governance framework. Validation, monitoring, governance oversight. | Does not address autonomy, learning, coordination, or emergence. Predates widespread agent deployment. |
| NIST AI RMF 1.0 | High | Comprehensive AI governance framework | Provides guidance on AI governance beyond model risk, including autonomy and governance. | Not yet adopted by regulators as official standard. Guidance is recommended, not required. |
| OCC Bulletin 2021-16 | Partial | Vendor and third-party AI management | Vendor governance; does not address internal agents. | Does not provide agentic-specific governance guidance. |
In banking and financial services, SR 11-7 is the primary model risk governance framework. Many banks apply SR 11-7 to all AI/ML systems, including agents. If SR 11-7 is insufficient for agents, banks may be under-governing agentic systems and exposing themselves to agentic-specific risks that the framework does not address.
Regulators are becoming aware of agentic systems and are evaluating whether SR 11-7 is adequate. As regulatory expectations evolve, banks may be required to implement governance beyond SR 11-7 for agentic systems.
Model Risk Conflation requires governance frameworks that extend beyond SR 11-7 to address agentic-specific risks. Our advisory engagements are purpose-built for banks, insurers, and financial institutions subject to prudential oversight.
Schedule a Briefing