Why customer support RAG hit a 65% accuracy ceiling
After two years of using RAG for customer support at a large product company, accuracy stopped improving at about 65%. The limit was confirmed with Ragas and manual reviews. The team tested standard RAG, , GraphRAG, , rerankers, several ways to split documents, and custom retrieval rules.
The main problem was not the technology itself but how customers ask for help. Real customers often type vague fragments such as “internet” or “doesn’t work” instead of clear, complete questions. A system that mainly searches for text with similar meaning struggles to identify the actual fault and the correct diagnostic steps from such input.
The technology supported a public , customer chat, an internal knowledge management system, and a Copilot app for support agents, with goals including faster calls, more first-contact solutions, and greater .
Key points
- Two years of RAG changes did not raise accuracy beyond about 65%.
- Ragas and manual reviews both supported the measured limit.
- Customers often send vague fragments rather than complete questions.
- Semantic alone is weak at diagnosing the cause of a problem.
- Support agents may need follow-up questions and step-by-step diagnostic logic.