Simple rules beat an LLM for many support ticket categories
An AI agent was sorting incoming customer tickets into about six groups so each ticket reached the right person. A , or LLM, handled the , but it sometimes sent a ticket to the wrong queue once or twice a week. Because the model appeared confident, the mistake was not noticed quickly, and the ticket could sit in the wrong place.
A closer look showed that most tickets were easy to classify from clear words such as “refund,” “down,” or “invoice.” The system was changed so simple handle the obvious cases, while the LLM only reviews tickets that do not match any rule. This cut wrong routes, reduced cost, and made each routing decision easier to explain to the client.
Key points
- The ticket-routing agent made wrong s about once or twice a week when the LLM handled .
- Most tickets could be sorted by obvious words like “refund,” “down,” or “invoice.”
- Simple now handle the clear cases before the LLM is used.
- The LLM is reserved for unclear tickets that do not match any rule.
- The change reduced cost, improved , and made decisions easier to explain.