A 40% token cost cut backed by quality checks
LLM usage grew, and the bill kept rising, so the were shortened and repeated work was handled with . Removing unnecessary instructions from bloated helped cut token spending by about 40%. The hard part was proving that the cheaper setup had not made the answers worse.
Shorter can quietly lower quality even when a few checked examples look fine. A prepared eval made the change measurable. Running the same ant inputs before and after the cost work showed that quality stayed steady, with one case slightly higher but likely just noise.
That made it possible to show leaders a number instead of asking them to trust a rough visual check.
Key points
- Shorter reduced unnecessary token use.
- helped avoid paying again for repeated work.
- Token spending dropped by about 40%.
- An eval compared quality before and after the changes.
- Manual spot checks were not enough to prove quality stayed the same.