Tensor split raised Qwen3.6-27B speed on two GPUs
in Q8_0 format reached about 100 on a two-card setup using an RTX 5090 and an RTX 3090 Ti. The same setup had been above 70 with layer split, but changing llama.cpp server to gave a large speed gain. lets both s work on the same parts of the calculation, instead of passing full layers between cards.
That helps when one card is much faster than the other. The workload was divided 70% to the RTX 5090 and 30% to the RTX 3090 Ti to better match their speed. Results changed by prompt, with some runs reaching up to 130 .
The tradeoff is power and heat: during generation, the two s alone used more than 750 watts. The setup also used a very large , two parallel jobs, 99 GPU layers, and memory-related llama.cpp options.
Key points
- Q8_0 ran across an RTX 5090 and an RTX 3090 Ti.
- Switching from layer split to raised speed from above 70 to about 100 .
- The was set to 70% on the RTX 5090 and 30% on the RTX 3090 Ti.
- Some prompts reached up to 130 .
- The two s used more than 750 watts during generation.