LLM judges need human checks before you trust the scores
scoring for LLM outputs can separate clearly good answers from clearly broken ones. The setup uses a dataset of inputs, lets another LLM grade each response for correctness and tone, and tracks the scores over time. A manual check of about 50 scored found a problem in borderline cases: the human score and the disagreed on about one third of them.
Some answers were technically accurate but missed the real point of the question, yet still received high scores. Some answers that seemed fine were marked down for tone in ways that were not easy to understand. A bigger risk is drift, because the judge is also a model.
If that model changes, gets updated, or is retired, the score may move even when the product itself has not changed. Regular comparison against and fixing the judge can make the scores more trustworthy.
Key points
- A manual check of about 50 found many disagreements in borderline cases.
- The handled obviously good and obviously bad outputs better than subtle ones.
- Answers that missed the point could still receive high scores.
- Tone scores can be hard to explain and may reject acceptable answers.
- and a fixed judge help keep scores useful.