For agents, fewer output tokens beat faster token speed
A local comparison of two models on the same agent work found that a model that thinks in fewer tokens can answer faster than one that generates tokens more quickly. used an average of 1,856 , while ThinkingCap-27B used 675. Both got 10 of 12 checkable answers correct.
Average response time fell from 30.1 seconds to 17.8 seconds, and long requests were up to 3.2 times faster. ThinkingCap-27B decoded more slowly, but it finished sooner because it produced far fewer tokens overall. On a separate test, tool-name accuracy rose from 0.089 to 0.179.
The tradeoff was weaker built-in knowledge: a lore-recall score dropped from 0.236 to 0.194, so agents that depend on model memory instead of may lose quality.
Key points
- fell from 1,856 to 675 on average with the same checked-answer score.
- Average response time dropped from 30.1 seconds to 17.8 seconds.
- Long prompts ran up to 3.2 times faster.
- Tool-name accuracy roughly doubled, from 0.089 to 0.179.
- Built-in knowledge recall got worse, which may hurt agents that do not use .