KV quantization still found facts in a 100k context

at q4_0 still found the right information inside a long input. The test used a , and the answer matched the inserted information. The also used q4_0 for its KV data.

The first example involved Harry Potter, so others pointed out that the answer might have come from rather than the long input. A follow-up used a less common line from an Italian 2026 book to reduce that concern.

Key points

  • at q4_0 still retrieved information from a .
  • The also used q4_0 KV data.
  • The Harry Potter example may have overlapped with .
  • A less common Italian 2026 book quote was used as a stronger check.
  • The main value is lower memory use and possible lower for agents.
Read original