Reducing hallucinations in a biography RAG system
A local RAG question-answering system is being built to answer only from a scholar’s biography PDF. The setup uses Ollama, , Mistral, embeddings, , and BM25.
The pipeline splits the PDF into chunks, searches for relevant chunks, sends the top chunks to a , and produces an answer. Retrieval seems reasonably good, but the model still invents facts that are not in the document.
It also sometimes becomes too cautious and says the text does not clearly state an answer even when the answer is present. The main question is whether the problem comes from , prompting, or the limits of smaller 7B to 8B models for biography-style question answering.
Key points
- The goal is a local question-answering system that answers only from one biography PDF.
- The pipeline is PDF chunking, embeddings, , top chunks, , then answer.
- The setup includes Ollama, , Mistral, BM25, and .
- The system sometimes invents facts even when retrieval seems reasonable.
- The open question is whether 7B to 8B models are strong enough for grounded biography question answering.