Very low model prices may point to heavier quantization
Some OpenRouter prices for large look hard to match with normal server costs. GLM-5.2 is the example. Even with a cheap 8xH200 spot setup at about $12 to $14 per hour and FP8, the math is tight.
If one node produces about 175 per second, it makes about 630,000 per hour. That equals about $22 per million before operations costs and profit. That is difficult to reconcile with API pricing near $4 per million unless throughput is much higher, is much cheaper, or the model is more heavily optimized or quantized.
Going below FP8 could reduce answer quality a lot, and even FP8 may reduce quality by an estimated 8% to 10%, though that kind of quality loss is hard to measure exactly.
Key points
- Large may be priced below what simple server cost math suggests.
- An 8xH200 setup at about $14 per hour and 175 per second implies about $22 per million before other costs.
- That is far above an API price near $4 per million .
- Possible explanations include higher throughput, cheaper , stronger , or heavier .
- For AI agents, lower token cost can be offset by lower quality and more retries.