A terminal tool for checking model-graded AI eval results
A terminal tool is available for reviewing answers that were graded by a model during AI evals. It splits a into claims, evidence, and verdicts.
It also highlights places where a verdict may not fit the evidence. This helped surface unusual cases that needed manual review.
A small terminal viewer is included for browsing the results. The code is on GitHub in the claim-memory-graph-sdk .
Key points
- The tool reviews answers in AI evals.
- It breaks a into claims, evidence, and verdicts.
- It flags cases where the verdict may not match the evidence.
- It includes a small terminal viewer for browsing results.
- The is named claim-memory-graph-sdk.