Biochemist builds new RAG engine, tests it on MuSiQue benchmark
A developer released a (RAG) engine built entirely from scratch rather than assembled from existing pieces. The creator holds a PhD in biochemistry and has worked in pharma and biotech for nearly two decades, and says the draws on biology concepts.
To test it, they used the same corpus as HippoRAG 2 — 11,656 Wikipedia passages — running the 1,000-question MuSiQue multi-hop benchmark and scoring the 496 questions that are answerable. The metric was SQuAD F1, a token-level /recall score computed without an .
Results were compared against BM25, LlamaIndex, and HippoRAG2. The full results, methodology, and limitations were published together, with AI assistance used to help write up the findings.
Key points
- New graph-based RAG engine built from scratch, not assembled from existing code
- Tested on the same corpus as HippoRAG 2 (11,656 Wikipedia passages)
- Scored on 496 answerable questions from the 1,000-question MuSiQue multi-hop benchmark
- Metric used: SQuAD F1, a score with no involved
- Compared against BM25, LlamaIndex, and HippoRAG2