R-ST-04 Organisational & Structural DAMAGE 2.8 / Moderate

Vendor Containment

Technology vendor fits "agent" label onto existing product without building genuine agent capabilities. Organization believes it has deployed agents when it has deployed automation with new branding.

The Risk

Agentic AI is a compelling concept and a valuable market opportunity. Technology vendors recognize this and respond by rebranding existing automation products as "agentic AI" without genuinely building agent capabilities. The organization believes it has deployed a true agent, but it has actually deployed automation with advanced marketing.

True agents have characteristics: they make decisions autonomously, adapt to new circumstances, delegate to other agents, and improve over time. Most existing automation tools do not have these capabilities. They execute predefined workflows, apply rules to data, and produce structured outputs. They are valuable, but they are not agents.

The vendor containment risk is that an organization chooses a "non-agent" tool because it is marketed as agentic. The organization then discovers that the tool lacks genuine agent capabilities: it cannot adapt to edge cases, it cannot learn from feedback, it cannot coordinate with other agents. The organization has made a suboptimal technology choice and faces re-platforming costs.

How It Materializes

An insurance company seeks to deploy an agentic system for claims processing. The company's requirements are: autonomous decision-making for routine claims, ability to handle edge cases by escalating to humans, continuous learning from feedback, and integration with existing claims systems.

The company evaluates three vendors. Vendor A provides a claims processing system that automates the application of claims rules. The system reads a claim, extracts key fields (claimant name, claim amount, loss type), looks up the policy in a database, checks coverage, and produces an approval or denial decision based on rules. The system includes advanced NLP for claim text analysis and a rules engine. It is marketed as "AI-powered claims agent."

Vendor B provides a low-code workflow platform with claims-specific templates. The platform lets the insurance company define claims workflows as sequences of steps. Each step can include decision logic. The platform includes some AI capabilities (predictive scoring, document classification). It is marketed as "intelligent automation."

Vendor C provides a true agent framework that lets the insurance company build agentic claims processing from scratch. The framework includes autonomous decision-making, online learning, multi-agent coordination, and planning. But it requires significant development effort and domain expertise to build.

The company chooses Vendor A because it is pre-built, requires minimal integration, and is marketed as agentic. The company implements the system and discovers, six months in, that: (1) the system has no true autonomy, making decisions based only on rules and escalating to humans when a rule does not cover a scenario; (2) the system does not learn from feedback, continuing to make the same mistakes even when claims handlers regularly override it; (3) the system operates in isolation and cannot coordinate with the company's underwriting, fraud detection, or provider network systems; and (4) the system can only be retrained through a lengthy vendor-supported process and cannot adapt autonomously to changing conditions.

The insurance company now faces a choice: invest more heavily in Vendor A's platform, accepting that it is not truly agentic, or switch to Vendor C, incurring re-platforming costs.

DAMAGE Score Breakdown

Dimension Score Rationale
D - Detectability 3 Vendor containment is not visible until the organization attempts to use the system for tasks that require true agent capabilities. Limitations become apparent during deployment or shortly after.
A - Autonomy Sensitivity 2 Vendor containment affects organizations deploying agents at high autonomy levels. Organizations using Digital Assistants or Apprentices (with human oversight) may not notice limitations because humans can work around them.
M - Multiplicative Potential 2 Vendor containment affects the specific technology choice. The organization may be locked into the vendor's limitations, but the impact is primarily the opportunity cost of not choosing a better solution.
A - Attack Surface 2 Vendor containment is not a direct security vulnerability. It is a procurement and vendor risk issue.
G - Governance Gap 3 Most organizations lack the expertise to evaluate whether a vendor's product is truly agentic or merely sophisticated automation with new branding. Procurement processes do not typically require assessment of agent capabilities.
E - Enterprise Impact 3 Vendor containment leads to suboptimal technology choices, lost productivity, and re-platforming costs if the organization switches to a better solution. Direct financial impact may be high (re-platforming) or low (accepting the limitations).
Composite DAMAGE Score 2.8 Moderate. Requires vendor due diligence and capability assessment before procurement.

Agent Impact Profile

How severity changes across the agent architecture spectrum.

Agent Type Impact How This Risk Manifests
Digital Assistant Low DA is human-facing and human-supervised. Limited agent capabilities may be acceptable. Humans can work around system limitations.
Digital Apprentice Low AP is supervised and learns gradually. Limitations are less critical because humans guide the learning process.
Autonomous Agent High AA must have genuine autonomous capabilities. Vendor containment (deploying non-agentic automation as an agent) will lead to system failure when the agent encounters edge cases that its rules do not cover.
Delegating Agent Medium DL invokes tools across systems. If the "agent" is not truly an agent but merely sophisticated automation, it will struggle to coordinate with other agents or tools.
Agent Crew / Pipeline High CR requires genuine agent capabilities in each agent in the pipeline. If any agent is merely automation with new branding, the pipeline will fail.
Agent Mesh / Swarm Critical MS requires all agents to have genuine agent capabilities (autonomy, learning, coordination). Vendor containment in any agent in the mesh will cause the mesh to fail.

Regulatory Framework Mapping

Framework Coverage Citation What It Addresses What It Misses
NIST AI RMF 1.0 Minimal N/A Framework-level guidance on AI governance. No guidance on assessing vendor claims or evaluating true agent capabilities.
EU AI Act Minimal N/A Focuses on high-risk AI applications, not vendor evaluation. No guidance on assessing vendor claims of agent capabilities.
ISO 42001 Partial Section 5 (AI Service Provider Governance) Recommends assessment of AI service providers. Does not specifically address agent capability assessment. No specific guidance on assessing agent capabilities.
OCC Guidance Partial Third-party management of AI/ML models Requires assessment of third-party vendor capabilities. Predates widespread agentic AI. No specific guidance on agent containment.
MAS AIRG Partial Section 3 (Risk Management) Requires firms assess AI system adequacy. Organizations should verify vendor claims. No specific guidance on vendor evaluation for agent capabilities.

Why This Matters in Regulated Industries

In banking and financial services, technology vendors play a critical role in implementing systems that handle customer money, credit decisions, and market operations. If a vendor makes misleading claims about system capabilities (for example, marketing automation as agentic), the bank may implement systems that are inadequate for their intended purpose. Regulators expect that banks choose technologies that are fit for purpose and that banks understand the capabilities and limitations of their systems.

In insurance, similar issues arise. Insurers depend on vendors for claims processing, underwriting, and fraud detection systems. If vendors oversell capabilities, insurers may deploy systems that are inadequate for complex decision-making.

In healthcare, medical device vendors and EHR vendors are subject to FDA and state regulatory oversight. Misleading claims about AI capabilities can lead to systems being approved or deployed inappropriately.

Controls & Mitigations

Design-Time Controls

  • Implement vendor capability assessment processes that go beyond marketing claims. Conduct technical due diligence: require vendors to demonstrate genuine agent capabilities (autonomous decision-making, online learning, multi-agent coordination) with proof-of-concept implementations.
  • Establish an agent capability checklist that defines what true agent capabilities look like: autonomy (decisions without human approval), adaptability (learning from feedback), coordination (working with other agents), planning (multi-step decision-making). Evaluate vendors against this checklist.
  • Conduct proof-of-concept testing with vendors before committing to large deployments. Test the system with edge cases that require true agent capabilities. If the system cannot handle edge cases autonomously, acknowledge that it is automation, not an agent.
  • Define capability requirements for your use case: be explicit about what agent capabilities you need. Evaluate vendors against your specific requirements.

Runtime Controls

  • Deploy capability monitoring that tracks whether the deployed system is demonstrating agent capabilities. Measure: autonomous decision rate, learning speed (improvement in accuracy from feedback), and coordination effectiveness.
  • Implement limitations documentation that explicitly states what the system cannot do. If the system is automation rather than a true agent, document this and adjust operational processes accordingly.
  • Establish vendor performance audits that compare vendor claims to actual system performance. If the system is not demonstrating claimed capabilities, escalate to vendor and consider re-platforming options.

Detection & Response

  • Monitor capability drift: if the system's demonstrated capabilities decline over time (due to vendor changes, version updates, or changing expectations), flag this for investigation.
  • Establish vendor accountability processes: if a vendor's system does not deliver on claimed capabilities, require the vendor to remediate or accept contract penalties.
  • Create a technology switching readiness plan: if vendor containment is discovered, plan for switching to a better solution. Maintain technical readiness to migrate to alternative vendors if containment becomes unacceptable.
  • Build agent capability libraries based on your evaluations: document which vendors provide true agent capabilities and which provide automation with agent branding.

Related Risks

Address This Risk in Your Institution

Vendor Containment requires rigorous due diligence and capability assessment that goes beyond what existing procurement frameworks provide. Our advisory engagements are purpose-built for banks, insurers, and financial institutions subject to prudential oversight.

Schedule a Briefing