NVIDIA’s compressed Nemotron model targets lower agent serving cost
-Labs-3-Puzzle-75B-A9B is a built for more efficient deployment. It is based on Nemotron-3-Super-120B-A12B and was compressed after training with a method called . The goal is to make interactive, reasoning-heavy, and workloads faster while keeping strong accuracy.
The model was reduced from 120.7 billion total parameters to 75.3 billion, and from 12.8 billion to 9.3 billion. Its design mixes MoE, Mamba, and attention layers, and it supports for faster text generation. NVIDIA says it delivers about 2 times higher server throughput on one 8xB200 node under matched user-throughput limits.
It also raises sustainable 1-million-token concurrency on a single H100 from 1 request to 8 requests. NVIDIA presents it as maintaining strong results on reasoning, coding, multilingual, , and agentic benchmarks.
Key points
- Nemotron-Labs-3-Puzzle-75B-A9B is a compressed version of Nemotron-3-Super-120B-A12B.
- Total parameters drop from 120.7 billion to 75.3 billion.
- drop from 12.8 billion to 9.3 billion, lowering the compute used per step.
- NVIDIA claims about 2 times higher server throughput on one 8xB200 node.
- Single-H100 1-million-token concurrency rises from 1 request to 8 requests in NVIDIA’s comparison.