Weak RAG can answer worse than using the full context
In a hard test, RAG that found only some of the needed evidence performed worse than giving the model all the material. Basic found about 42% of the required paragraphs, and answer accuracy was 0.22. Giving the model all 20 paragraphs without retrieval reached 0.47 accuracy.
The reason is that multi-hop questions need a full chain of facts; if one link is missing, the final answer often fails. When the model received the exact needed paragraphs through , accuracy rose to 0.66. The biggest room for improvement was retrieval completeness, not asking the model many times and voting on the answers.
raised per-paragraph recall to 0.72, but it still did not beat the full-context setup. Self- and multi-model voting also did not fix the missing-evidence problem.
Key points
- Basic found about 42% of the needed paragraphs and reached 0.22 accuracy.
- Using all 20 paragraphs without retrieval reached 0.47 accuracy.
- reached 0.66 accuracy by giving the model the exact needed evidence.
- Multi-hop questions often fail when even one required fact is missing.
- Retrieval completeness mattered more than answer voting or repeated sampling.