Agent memory should be bounded state, not full transcript replay
Most default to : feeding the whole conversation, or retrieved chunks of it, back into the every turn. This works fine for short tasks but breaks down on long ones, and simply enlarging the does not fix it.
Chroma's context-rot report evaluated 18 models, including GPT-4.1, Claude 4, Gemini 2.5, and Qwen3, and found accuracy degrading well before hitting the token limit, sometimes dropping 30-50% even on simple retrieval or replication tasks. Position within the context also matters: information at the start and end of the window survives, while content in the middle tends to degrade.
A recent paper, "AI Agents Need Memory Control Over More Context" (arXiv 2601.11653), argues agents should maintain a bounded internal state that gets committed each turn, rather than an ever-growing transcript. It explicitly separates recalling an artifact from committing it to , and claims lower drift and than or retrieval across IT-ops, security, and healthcare workflows, though without hard benchmark numbers.
Key points
- Most default to , re-feeding the full conversation into the each turn.
- Chroma's context-rot report: across 18 models, accuracy drops 30-50% well before the token limit, with middle-of-context information especially fragile.
- A new paper proposes a bounded internal state committed each turn instead of a transcript, separating recall from committing to memory.
- Enlarging the isn't a real fix — deciding what NOT to keep in memory is framed as the key design decision.