When an LLM judge passes an answer that humans find unsafe
An can approve a response while a er sees the same response as unsafe. The issue is not always that the ignores the written rubric.
The deeper problem is that the rubric may miss subtle human judgment. This can include a small policy violation, wording that technically follows the rules but would upset a real customer, or an answer that is correct but wrong for the situation.
Putting a human on every would be too costly and slow, which weakens the reason to use an in the first place. The practical question is how to reduce the gap when the and ers disagree on safety.
Key points
- An may pass a response that a er considers unsafe.
- The written rubric may not capture subtle risks that humans notice.
- Risky cases include small policy violations, upsetting tone, or correct answers used in the wrong context.
- for every is too expensive and slow for many teams.
- Teams need a way to the against human safety judgment.