A hard RAG test with a long Supreme Court ruling
The Supreme Court birthright citizenship ruling in Trump v. Barbara is a 194-page document. It includes a Roberts majority opinion, a Jackson concurrence, a Kavanaugh opinion that partly agrees and partly disagrees, and three separate dissents.
The phrase “subject to the jurisdiction” changes meaning depending on which opinion it appears in. That makes the document difficult for simple search systems that split text into chunks and match similar wording. handled it with a , approach.
Its strongest result was finding the right opinion and section when comparing Kavanaugh’s with the majority’s . The answers also pointed to exact blocks and pages, so the PDF could be checked directly.
Key points
- A 194-page Supreme Court ruling was used as a difficult test.
- Several opinions used the same phrase with different meanings.
- separated the relevant opinions and sections instead of mixing similar text together.
- Each answer pointed back to specific pages and blocks in the PDF.
- Better can help use fewer tokens and give more checkable answers.