Can RAG beat frontier LLMs on data everyone can already access?
A firsthand account describes building and running a system over a corpus of 10,000 medical papers at a time, at a small startup doing medical literature analysis. The team experimented with combining Graphiti, a knowledge-graph tool, with UMLS, a standardized medical terminology database, but the ultimately used a simple, vanilla RAG setup.
After running this for a client over a couple of months, the client judged the tool inferior to ChatGPT for two reasons: response speed was slow, taking 2-3 minutes per answer, and the output didn't feel meaningfully more medically accurate or capable than ChatGPT's. This experience raises a broader question: on public data, can RAG techniques actually outperform that were already trained on that same public data, even when a small team focuses narrowly on a niche topic?
Key points
- Built a vanilla RAG system over corpora of 10,000 medical papers at a time
- Tried combining Graphiti (a tool) with UMLS (a standard medical terminology database), but production used simple RAG
- A client judged the tool inferior to ChatGPT after a couple months of use
- Response time was slow, taking 2-3 minutes per query
- Raises the open question of whether RAG can beat on data the models already trained on