tunelab moves repeat LLM work from APIs to local small models

tunelab is an tool for moving repeat LLM tasks, such as , routing, and data extraction, away from expensive s and onto small models that run locally. It fine-tunes those small models on your own data, then checks whether they beat the API on held-out test data before you ship them. On the Banking77 test, a free local classifier reached 88.5% accuracy, while reached 81.8% on the same task.

A 3-tier cascade reached 94% accuracy, handled about 88% of traffic locally, and cut cost by 8x compared with sending everything to a . The tool works by trying options from cheapest to most expensive. It starts with a better prompt or cheaper model, then tries embedding similarity, a small classifier, LoRA fine-tuning, and only rarely continued pretraining.

It stops as soon as one level clears the required accuracy bar, so fine-tuning is used only when it is worth it.

Key points

  • tunelab targets repeat LLM work such as , routing, and extraction.
  • It tests local small models against API results before deployment.
  • On Banking77, the local classifier reached 88.5% accuracy versus 81.8% for .
  • A 3-tier cascade reached 94% accuracy, served about 88% of traffic locally, and reduced cost by 8x.
  • The workflow tries cheap fixes first, then moves up to LoRA fine-tuning only if needed.

Sources covering this story (2)

Read original