Agent memory needs more than trusted-looking sources
cannot safely trust saved memory or retrieved documents just because the source looks d. An core tested four defenses: scoring records by value, requiring , letting newer records replace older ones in a fixed way, and giving credit to records that seemed to lead to success. All four failed when the attacker knew how the defenses worked.
The weak point was that the record itself supplied the signals used to judge it, so the attacker could write fake value, fake support, convenient timing, or fake success. and cost are harder for the writer to fake, but only proves where something came from, not whether it is true. False memory can still enter through a real, d session and pass a source check.
Related tests point to the same pattern: RAG can quote the right text while pointing to the wrong source about 30% of the time, source and metadata can be forged, and a cached fact can become stale even though it still fits the context perfectly. The safer rule is to make memory cheap to store but expensive to use for broader influence, by requiring separate anchored evidence before it shapes an answer outside its original scope.
Key points
- Four common memory defenses failed against an attacker who knew the rules.
- proves the source path, not the truth of the content.
- RAG can return correct text while attaching the wrong .
- Source and metadata are weak if the writer can fill them in.
- Memory can cut , but important uses need separate evidence checks.
Sources covering this story (7)
- r/RagAgent memory needs more than trusted-looking sources ↗
- Hacker NewsShow HN: Halo – open-source, tamper-evident runtime evidence for AI agents ↗
- r/LLMDevsI stopped trusting LLM-inferred memory after it poisoned my sessions — here's the evidence-gated redesign, plus the two harness primitives it pairs with ↗
- r/RagMy RAG quotes the right text but cites the wrong source ~30% of the time :/ ↗
- r/RagYour RAG's "source" and "corroboration" metadata don't stop memory poisoning — the attacker writes those fields too (10-model test + runnable probe) ↗
- r/RagWe tried to poison our own RAG store — the retrieval-time defenses didn't generalize ↗
- r/LLMDevsThe agent failure mode no eval catches: acting on a fact that was true when it was cached and wrong when it was used ↗