Local LLM cost math should include input speed, not just output
Local LLM return on investment is often judged by decoding speed, which is how fast the model writes an answer. For agent work that reads a lot of material, prefill speed can also change the economics because it measures how fast the model processes input before writing. In one example, GLM 5.2 running on four machines with 4-bit , , and other tuning reached about 60 per second with 6 concurrent users.
If that ran as a 24/7 agent workload, it would produce about 5.18 million per day, or about $22 per day at $4.40 per million . The same setup is said to reach about 3,000 per second during prefill. are cheaper at about $1.40 per million for GLM 5.2, but prefill can be 10 to 30 times faster than decoding, and about 50 times faster in this example.
That means output speed alone may understate the value of local hardware for agents that read large inputs.
Key points
- Local LLM return on investment is often discussed through decoding speed only.
- The GLM 5.2 example used four machines and reached about 60 per second with 6 concurrent users.
- At 24/7 use, that equals about 5.18 million per day, or about $22 per day at the stated output price.
- Prefill on the same setup is said to reach about 3,000 per second.
- Input-heavy agent workloads should include prefill speed when judging local hardware value.