RAG chatbots may improve more from better retrieval than new models
A firsthand build found that many chatbot failures came from finding the wrong supporting material, not from the itself. The first version used only, then added with BGE and Qdrant, BM25, RRF, and CrossEncoder reranking. Those additions helped, but the biggest practical gain came from using URL structure and metadata to guide retrieval.
For a college chatbot, content can be grouped into admissions, departments, exams, scholarships, notices, and student services. When someone asks about scholarships, the system can search scholarship material first instead of scanning every document. This reduces irrelevant results, helps the system find more useful material, and gives the reranker better candidate text to work with.
The main lesson is that retrieval design can matter more than switching to a different model. An example project is available in the omnichannel-rag-chatbot.
Key points
- Many chatbot failures can come from poor retrieval, not the .
- The system moved beyond by adding , BM25, RRF, and CrossEncoder reranking.
- Grouping content by topic lets the chatbot search the most relevant area first.
- Metadata can reduce noise before expensive happen.
- Better retrieval design may improve results more than changing models.