AnythingLLM gives inaccurate document answers even with RAG
was run with on a computer with 128GB of memory. Its was set to about 128,000 tokens, and several Markdown files were placed directly into the chat. Answers about those files were still highly inaccurate.
Switching to RAG, which retrieves relevant document sections before answering, did not improve the results. The cause and a working for accurate answers remain unknown.
Key points
- and ran on a computer with 128GB of memory.
- The was set to roughly 128,000 tokens.
- Several Markdown files were loaded directly into the chat.
- Answers about the files were highly inaccurate.
- RAG mode did not fix the accuracy problem.