Long documents may work better as maps, not chunks
A local pipeline for mixed documents such as Markdown files, PDFs, and research papers can work reasonably well for simple question answering with standard text splitting and . The problems start when the same setup is connected to reasoning . If a contract or API document is cut into 512-token pieces, the agent may receive a useful paragraph but miss the parent section, the document , and how that paragraph fits into the whole file.
Overlapping chunks and parent-child can help a little, but they often add more text to the without solving the structure problem. A different approach is to map the document first. The model indexes metadata, , and token ranges for each section, then the agent searches that map, finds the right document, checks the outline, and asks for the exact section it needs.
This gives the agent a instead of a pile of pre-cut snippets. For long files where structure matters, this keeps more meaning while using context more carefully.
Key points
- Standard 512-token chunking can strip away the structure of long documents.
- Reasoning need to know where a passage sits in the wider document.
- Overlapping chunks and parent-child may increase usage.
- Mapping metadata, , and token ranges lets the agent request only the needed section.
- The approach is most useful for long documents where sections and outlines carry meaning.