Agent task cost fell about 70% by moving routine calls off GPT-4o
An internal research agent needed about 40 to 60 calls for each task. Using GPT-4o for all of them cost more than $1 per task, even though many calls were only for routing or extracting information. Kimi K2.7, GLM 5.2, Qwen 3.7 max, and DeepSeek V4-Pro were tested on the same evaluation set, with about 300 runs each, using standard tool prompts and no model-specific tuning.
Kimi K2.7 handled well, including parallel tool use and following the required schema. GLM 5.2 stayed usable when given very of about 60,000 to 80,000 tokens. Qwen 3.7 sometimes returned tool arguments as a JSON string instead of an object when under load, which a retry wrapper could fix.
Deeply nested schemas with optional fields caused more trouble for every than for GPT-4o, so flatter schemas worked better. The final setup used Kimi for tool steps, GLM for synthesis, and GPT-4o only for the final answer shown to the user, cutting cost from about $1.10 to about $0.35 per task.
Key points
- The agent made 40 to 60 model calls per task, so small per-call savings added up quickly.
- Kimi K2.7 looked strong for in this test.
- GLM 5.2 worked well for synthesis with very .
- struggled more with deeply nested schemas than GPT-4o.
- Endpoint testing mattered because latency varied heavily between providers serving the same model.