Old P40 GPU runs a 35B model at 80 tokens/sec
Someone ran the Qwen3.6 35-billion-parameter model on a single Nvidia P40 (24GB VRAM), using a format called UD Q4_K_M, and reached a burst speed of 80 . They used , a modified fork of llama.cpp, and fit a 100,000-token context (the amount of conversation the model can remember at once) on this single card. That speed is only a short burst, not a sustained rate.
To keep the model reliable enough for agent use, they stopped short of applying an even more aggressive level called Turbo2. They also disabled the model's reasoning (thinking) mode because it kept getting stuck repeating the same thoughts in a loop. Skipping the vision (image-understanding) component saved about 300MB of memory, which was enough to fit the full 100,000-token context on the P40.
The exact command and settings used, including batch size, GPU layer count, and thread count, were shared as well.
Key points
- A single Nvidia P40 (24GB VRAM) ran a 35B-parameter model at a burst 80 tokens/sec
- Used , a modified llama.cpp fork, with UD Q4_K_M
- Dropped the vision component to save VRAM and fit a 100k-token context
- Reasoning mode was disabled due to repetitive thinking loops