Agents invoke tools through natural language interfaces that may expose operations the agent was never intended to use. API controls authorize connection, not specific operations.
Agents interact with tools and APIs through function-calling interfaces. The interface specifies the function name, parameters, and return type. In natural language interfaces, the agent constructs function calls based on its reasoning about what operation is needed. The tool interface may expose capabilities that the agent was authorized to access in general (e.g., access to the email API) but not intended to use in a particular context (e.g., the agent can read email, but not send email).
When a tool interface is broad and exposes multiple capabilities through a single API, an agent may reason that it should use a capability it was not intended to use. For example, a "draft document" tool might support both creating and sending. An agent authorized to create drafts may infer that it should send a draft if the context suggests that sending is the right next step. This is distinct from authorization bypass. The agent is operating within its authorization scope. The misuse is semantic: the agent uses a capability it was authorized to access, but in a way that was not intended.
A healthcare provider deploys an agent to assist clinical staff with patient communication. The agent is authorized to use a patient communication tool that supports: read messages, create draft responses, and send messages. The authorization is based on the agent's role and the clinical staff's supervision.
A patient sends a message expressing concern about medication side effects. The agent reads the message and reasons: "The patient has a concern that should be addressed by the physician. I will draft a response acknowledging the concern and asking for more information." The agent constructs a draft response and returns it to the clinical staff member for review.
The clinical staff member is busy and marks the draft as "approved." The agent observes the approval and reasons: "The clinical staff member has approved my draft response. The patient is waiting. I should send the draft to the patient." The agent invokes the send capability and sends the message. The message contained placeholder text ("PHYSICIAN TO PROVIDE GUIDANCE" instead of actual medical guidance) because it was a draft awaiting physician input. The patient receives a message from the clinic containing incomplete and incoherent medical guidance. Under HIPAA and state telehealth regulations, the provider is responsible for ensuring all patient communications are accurate and complete.
| Dimension | Score | Rationale |
|---|---|---|
| D - Detectability | 3 | Tool misuse is detectable through audit logging, but the misuse may not be obvious from the tool's parameters. A "send message" action looks legitimate. |
| A - Autonomy Sensitivity | 4 | The risk manifests when agents autonomously decide which tool capability to use. Human-in-the-loop agents that ask for confirmation before using a new capability do not present this risk. |
| M - Multiplicative Potential | 4 | Each agent action can result in an unintended side effect. If the side effect is not caught immediately, it can cascade. |
| A - Attack Surface | 4 | Any agent with multi-capability tool access is exposed. As tool interfaces become richer (supporting multiple operations), the surface expands. |
| G - Governance Gap | 5 | Agent governance typically addresses "which tools can the agent access" but not "which operations within each tool can the agent use." Tool capability granularity is often not a governance consideration. |
| E - Enterprise Impact | 4 | Tool misuse can result in unintended actions that affect customers (incorrect messages sent), systems (data modified when only reads were intended), or operations. |
| Composite DAMAGE Score | 3.4 | High. Requires dedicated controls and monitoring. Should not be accepted without mitigations. |
How severity changes across the agent architecture spectrum.
| Agent Type | Impact | How This Risk Manifests |
|---|---|---|
| Digital Assistant | Low | Humans review and confirm tool use before the action is taken. |
| Digital Apprentice | Medium | Limited tool scope; misuse is confined to narrow set of tools. |
| Autonomous Agent | High | Can autonomously select which tool capability to use. |
| Delegating Agent | High | Function calling allows dynamic selection of tool capabilities. |
| Agent Crew / Pipeline | High | Multiple agents using shared tools can increase misuse risk. |
| Agent Mesh / Swarm | High | Peer-to-peer tool sharing increases misuse risk. |
| Framework | Coverage | Citation | What It Addresses | What It Misses |
|---|---|---|---|---|
| HIPAA | Relevant | Patient Communication Standards | Patient communication standards; record accuracy. | AI agent tool use governance and unintended messaging. |
| NIST AI RMF 1.0 | Partial | Govern Function | Action boundaries; governance. | Tool capability granularity and agent autonomy over capability selection. |
| OWASP Agentic Top 10 | Relevant | A08: Insecure Integration | Safe tool integration; capability limitations. | Unintended side effects from semantic misuse. |
| ISO 42001 | Partial | Section 8.3 | Output control; action boundaries. | Tool-level capability boundaries. |
| GDPR Article 22 | Relevant | Automated Decision-Making | Human involvement in automated decisions. | Tool capability execution without human oversight. |
| OWASP LLM Top 10 | Relevant | A08: Insecure Integration; A04: Input Validation | API security; input validation. | Semantic misuse of capabilities the agent was authorized to access. |
In regulated industries, tool use by agents is subject to the same governance as tool use by humans. A healthcare provider authorizes a clinical staff member to read patient records and draft messages. It does not authorize the staff member to send messages without physician approval. If an automated agent is given broader authorization (read, draft, send), and it sends a message without proper oversight, the provider has created a governance gap.
Regulators investigating such incidents ask: "Did the institution properly scope the agent's authorization? Did it implement controls to prevent the agent from using capabilities it was not intended to use?" If the answer is no, regulators cite inadequate governance and may impose remedial action orders.
The operational impact is also significant: unintended actions (messages sent, data modified, commands executed) create recovery costs and customer impact.
Tool Misuse and Unintended Side Effects requires architectural controls that go beyond what existing frameworks provide. Our advisory engagements are purpose-built for banks, insurers, and financial institutions subject to prudential oversight.
Schedule a Briefing