vLLM bug cut Qwen 3.6 quality under production load
Running with vLLM and FP8 produced much worse answers than llama.cpp versions, especially when many requests arrived at once. Quality benchmark scores fell by about 20 percentage points in absolute terms. The cause was a vLLM bug that could corrupt results when and were enabled together for or 35B.
Applying the patch in vLLM 48375 resolved the observed quality loss. The affected setup used Qwen/-FP8, and the difference became clear when it was compared with other s.
Key points
- and 35B may produce corrupted results when and are used together.
- Quality scores dropped by about 20 percentage points when the system was under heavy concurrent load.
- The patch in vLLM 48375 resolved the reported quality loss.
- Test both answer quality and speed with realistic numbers of simultaneous requests before deploying an AI agent.