A long-context model tracked sources but weakened after 200K tokens
A real Q3 strategy brief was tested with using 14 sources. The input included PDFs, earnings call , and two analyst notes, totaling about 340,000 tokens. The goal was to create one ument while keeping a clear map between each claim and its source.
Source tracking stayed useful across the long input. The model could separate whether a claim came from a Gartner note or a earnings call without extra prompting. The weak point appeared after roughly 200,000 tokens.
Below that level, caveats and limits stayed visible; above it, the model tended to smooth over conflicts instead of marking them clearly. ’s stated 83.5 result and ICLR example may show other strengths, but they do not directly prove on long business synthesis work.
Key points
- The test used 14 sources and about 340,000 tokens of real Q3 strategy material.
- kept source links clear across the long input.
- After about 200,000 tokens, caveats became less reliable.
- The model sometimes blended conflicting evidence instead of calling out the conflict.
- Benchmark claims like and ICLR examples may not match real long-document synthesis work.