KV quantization still found facts in a 100k context
at q4_0 still found the right information inside a long input. The test used a , and the answer matched the inserted information. The also used q4_0 for its KV data.
The first example involved Harry Potter, so others pointed out that the answer might have come from rather than the long input. A follow-up used a less common line from an Italian 2026 book to reduce that concern.
Key points
- at q4_0 still retrieved information from a .
- The also used q4_0 KV data.
- The Harry Potter example may have overlapped with .
- A less common Italian 2026 book quote was used as a stronger check.
- The main value is lower memory use and possible lower for agents.