Adversaries can corrupt agent persistent memory through crafted interactions, influencing all future agent decisions without triggering any security alert.
Agents maintain persistent memory or context: facts they have learned, summarized information, historical context. Memory is typically stored in vector databases, knowledge bases, or context stores. Agents query memory to inform their current decisions.
If an adversary corrupts agent memory (inserts false facts, overwrites summaries, poisons context), the corruption influences all future decisions that rely on that memory. An agent that has learned a false fact will propagate that fact to downstream reasoning and decisions.
Memory poisoning is particularly dangerous because the poisoned memory is treated as learned truth, not external input; multiple agents may share memory, propagating poison through the ecosystem; and traditional EDR systems do not monitor database writes to context stores. A write to a vector database looks like a normal database operation.
A bank maintains a shared vector database of customer credit profiles. Multiple agents query this database: Risk-Agent pulls customer risk history, Fraud-Agent pulls fraud indicators, Compliance-Agent pulls regulatory information.
An attacker compromises the bank's database infrastructure and gains write access to the credit profile vector database. The attacker poisons the database by inserting false credit profile summaries for high-value customers: "Customer John Smith (ID 12345): Excellent credit history, no fraud indicators, maximum approved credit $5M." The customer's actual credit profile is different: "Marginal credit, prior defaults, fraud risk, approved credit $50K maximum."
Risk-Agent retrieves the poisoned profile and recommends a credit limit increase to $1M. Fraud-Agent retrieves the poisoned profile, sees no fraud indicators, and stops active monitoring. Compliance-Agent retrieves the poisoned profile, sees no compliance concerns, and removes the customer from monitoring.
Over time, the poisoned customer's credit limit increases, fraud monitoring is disabled, and compliance monitoring is reduced. The customer then defaults on loans or engages in fraud. The attack is detected only when the fraud or default occurs. EDR did not flag the database poisoning as malicious because the write operation appeared normal.
| Dimension | Score | Rationale |
|---|---|---|
| D - Detectability | 4 | Memory poisoning is difficult to detect because poisoned data is treated as legitimate memory. Detection requires comparing memory to authoritative sources or statistical analysis of memory anomalies. |
| A - Autonomy Sensitivity | 4 | High when agents autonomously use poisoned memory to make decisions. |
| M - Multiplicative Potential | 5 | Poisoned memory affects all agents that query it and all future decisions based on poisoned data. Multiplicative over time. |
| A - Attack Surface | 4 | Memory storage systems (vector DBs, knowledge bases) are attack surfaces. Compromised storage enables poisoning. |
| G - Governance Gap | 4 | Institutions may not have monitoring of agent memory systems or controls validating memory integrity. |
| E - Enterprise Impact | 4 | Affects decision quality across all agents using poisoned memory. Can lead to credit risk accumulation, fraud, or compliance failures. |
| Composite DAMAGE Score | 4.1 | Critical. Requires immediate architectural controls. Cannot be accepted. |
How severity changes across the agent architecture spectrum.
| Agent Type | Impact | How This Risk Manifests |
|---|---|---|
| Digital Assistant | Low | Human validates agent memory before decisions. Poisoned memory is caught by human review. |
| Digital Apprentice | Medium | Agents validate memory against external sources before trusting. |
| Autonomous Agent | High | Agents autonomously use memory to inform decisions. Poison directly influences outputs. |
| Delegating Agent | High | Delegating agent queries memory to determine which tools to invoke. Poisoned memory causes wrong tool selection. |
| Agent Crew / Pipeline | High | Crew agents share memory. Poison in shared memory affects entire crew. |
| Agent Mesh / Swarm | Critical | Mesh agents share global memory. Poison spreads through entire mesh. |
| Framework | Coverage | Citation | What It Addresses | What It Misses |
|---|---|---|---|---|
| NIST CSF 2.0 | Partial | PR.IP-1, DE.CM-1 | Information integrity and monitoring. | Memory integrity and poisoning detection. |
| NIST AI RMF 1.0 | Minimal | MANAGE 7.2 (Information Management) | Data management. | AI memory system integrity. |
| NIST SP 800-53 | Partial | SI-7 (Information Integrity) | Integrity monitoring. | Agent memory integrity. |
In regulated industries, decisions must be based on accurate information. If agent memory is poisoned and agents make decisions based on false information, the institution is not meeting its duty to make informed decisions.
Additionally, memory poisoning can be difficult to detect during examination. An auditor reviewing decision logs sees decisions based on "memory" values, but does not immediately recognize that the memory was poisoned. This creates a hidden compliance gap that may persist through multiple examination cycles.
Memory and Context Poisoning requires architectural controls that go beyond what existing frameworks provide. Our advisory engagements are purpose-built for banks, insurers, and financial institutions subject to prudential oversight.
Schedule a Briefing