LLM judging matched human reviewers only about 71%
On the same 400 outputs, three ers agreed with each other about 89% of the time, while an LLM judge matched the ers about 71% of the time. The LLM judge marked too many borderline but acceptable outputs as bad, and it missed some subtle real problems that humans caught. Asking the judge to write its reasoning before giving a score raised agreement to about 76%.
Adding a few examples of borderline cases raised it to about 78%. Running three LLM judges and using a raised it to about 82%. a smaller on human-labeled data reached about 85%, but it added ongoing work.
Since ers only reached about 89% agreement with each other, the practical question is whether closing the remaining 7-point gap from 82% to 89% is worth the extra cost.
Key points
- The test used 400 outputs reviewed by both humans and an LLM judge.
- ers agreed with each other about 89% of the time.
- The LLM judge matched humans about 71% of the time at first.
- Prompt changes improved agreement to about 76% and 78%.
- Three judges with reached about 82%, while reached about 85% with more upkeep.