The real agent speed boost came from system design, not a faster model
The biggest gain in making an AI agent feel faster came not from switching models but from restructuring how the system works around the model. Running independent at the same time instead of one after another cut wait time significantly. Caching data that rarely changes avoided repeated lookups.
Sending less context to the model reduced processing time. For long-running tasks, returning partial results immediately while the rest finished in the made the experience feel snappier. In some spots, an LLM call was replaced entirely with plain code.
None of these changes moved benchmark scores, but they made the agent noticeably faster to actually use.
Key points
- Running independent in parallel instead of sequentially improves felt speed
- Caching rarely-changing data avoids repeated lookups
- Sending less context to the model speeds up
- For long tasks, showing partial results early while the rest in the improves the experience
- Replacing some LLM calls with plain code cuts both latency and cost