DiffusionGemma 26B is fast on a 4090 but has real limits
26B ran on an Nvidia 4090 and reached 475 on the first request. Depending on answer length and , speed ranged from about 290 to 700 , with long answers coming out especially fast. Setup was not simple: it needed a custom vLLM and a Gemma tool and parser.
Since 3090 and 4090 cards cannot use nvfp4 here, the test used -26B-A4B-it-AWQ-INT4. The model is heavy, so is limited; an 8,000-token context worked, but much longer use looked unrealistic. It also works best for one user at a time because batching multiple requests slows it down.
Answer quality was worse than the regular Gemma 26B A4B model, and it struggled to find small details inside . For short prompts, the first token arrived slightly slower than with a regular .
Key points
- 26B reached about 290 to 700 on a 4090.
- It required a custom vLLM Docker setup and a Gemma tool and parser.
- Long outputs were fast, but handling was weak.
- Batching slowed it down, which makes it poor for many users or many agents at once.
- The regular Gemma 26B A4B model gave better answers in this comparison.