Cost of out-of-policy actions, runtime errors, and mission failures accumulates without systematic measurement or root cause analysis.
Every time an agent makes an error, there is a cost: customer complaint handling, manual correction, reputation damage, regulatory escalation (depending on the context). When errors are small and scattered, the cost is invisible. When errors accumulate, the total cost can exceed the value of the automation.
In Six Sigma, this is called "cost of poor quality" (COPQ). COPQ includes: cost of rework (fixing errors), cost of prevention (measures to prevent errors), and cost of failure (customer impact, regulatory penalties).
Organizations deploying agents often do not measure COPQ systematically. They track high-level metrics (agent availability, throughput) but not the cumulative cost of errors. As errors accumulate, COPQ rises, and the ROI of the agent deployment declines.
A financial services firm deploys an agentic document processing system to extract structured data from customer statements (account balance, transaction history, etc.). The agent is designed to process documents at 10x the speed of manual data entry and at 95% accuracy.
In the first month, the agent processes 10,000 documents and produces 500 errors (5% error rate). The firm's back-office team manually reviews the agent's extractions for quality assurance (a sample of 500 documents). They identify 20 errors in the sample, which is consistent with the 95% accuracy target. They approve the deployment.
In the subsequent months, the agent processes 50,000 documents total. The back-office team does not continue to do 100% review (too expensive). Instead, they do spot checks and complaint investigation. Complaints come in: "The agent extracted my balance as $5,000 when it is actually $50,000," "The agent missed a transaction," etc. The team handles these complaints reactively, correcting each error manually.
By month 6, the cumulative cost of errors is significant: 50,000 documents times 5% error rate equals 2,500 errors. Even if only 10% of errors generate complaints and require manual correction (250 errors), the correction time at 10 minutes per error is 2,500 hours of labor. This is equivalent to 1.2 full-time staff dedicated to fixing agent errors.
Additionally, the errors are causing customer friction. Some customers receive incorrect statements or account summaries based on the agent's extractions. The firm's compliance team is concerned that errors in financial statement extraction could constitute regulatory violations if they affect customer decisions.
When the firm calculates the total cost of poor quality, it discovers that the agent deployment is nearly break-even: labor savings from automation are offset by the labor required to fix errors and handle complaints. The deployment is not generating positive ROI.
| Dimension | Score | Rationale |
|---|---|---|
| D - Detectability | 3 | Defect accumulation is detectable through COPQ tracking, but many organizations do not systematically measure COPQ. |
| A - Autonomy Sensitivity | 2 | Both autonomous and supervised agents generate defects. |
| M - Multiplicative Potential | 2 | Defects accumulate linearly (each error adds cost), not exponentially. But cumulative cost grows with scale. |
| A - Attack Surface | 4 | Any agent generating errors is exposed. Most agents have nonzero error rates. |
| G - Governance Gap | 4 | Agent governance does not mandate systematic COPQ measurement and tracking. |
| E - Enterprise Impact | 3 | Accumulated defect costs reduce ROI and may make agent deployment unviable economically. But this is a financial issue, not a regulatory or safety issue. |
| Composite DAMAGE Score | 2.5 | Moderate. Requires systematic COPQ tracking and ROI monitoring for all agent deployments. |
How severity changes across the agent architecture spectrum.
| Agent Type | Impact | How This Risk Manifests |
|---|---|---|
| Digital Assistant | Low | Humans catch and correct obvious errors. |
| Digital Apprentice | Medium | Limited scope; defect cost is bounded. |
| Autonomous Agent | High | Errors accumulate without human interception. |
| Delegating Agent | High | Errors in tool invocation and delegation. |
| Agent Crew / Pipeline | High | Errors in one agent propagate to the next. |
| Agent Mesh / Swarm | High | Distributed errors across the mesh. |
| Framework | Coverage | Citation | What It Addresses | What It Misses |
|---|---|---|---|---|
| NIST AI RMF 1.0 | Partial | Performance monitoring and optimization | Performance measurement. | Cost of poor quality and defect accumulation. |
| ISO 42001 | Partial | Section 8.5, Performance monitoring and optimization | Performance optimization. | COPQ as a metric. |
While COPQ is primarily a business efficiency metric, it matters in regulated industries because accumulated defects can lead to regulatory violations. If an agent is generating defects at a high rate and the organization does not fix them, eventually the defects will cause customer harm or compliance violations. The organization's failure to measure and address COPQ is a governance failure.
Additionally, if an agent is not economically viable (COPQ exceeds savings), the organization should not deploy it. Deploying agents that do not produce value is a misallocation of resources and a sign of poor governance.
Defect Waste Accumulation erodes the business case for agentic AI. Our advisory engagements help institutions implement systematic COPQ tracking integrated with agent governance.
Schedule a Briefing