Open-weight model GPU cost benchmarks are being requested
HexGrid Cloud is offering to test open-weight chat AI models on different GPUs. The goal is to see how fast and cheaply these models can run when many requests happen at the same time. The suggested models include Nemotron-3 Super 120B-A12B, Nemotron-3 Nano 30B A3B, Qwen-3.6 27B, Llama 3.3 70B Instruct, Gemma-4 31B, and Devstral-Small-2-24B-Instruct-2512.
Other open-weight chat models can also be suggested if they fit on one H200 with 141GB of memory. The available GPUs are , L40S, H100, and H200, with FP8, AWQ, or BF16 as compute choices. can be set to 8K, 32K, 64K, or 128K tokens.
The planned measurements include , , time per , performance under many users, and cost per million tokens. HexGrid Cloud says it will publish the settings and flags so the tests can be repeated.
Key points
- Open-weight chat models will be tested on several GPU options.
- The tests are meant to reflect real use with many requests happening at once.
- can go as high as 128K tokens, which matters for agents that read long material.
- Planned results include speed, delay before the first token, throughput, and cost per million tokens.
- The test settings are expected to be shared so others can reproduce the results.