Choosing real-world tests for Qwen3.6 27B quantized models
The goal is to compare several quantized versions of in real use, not just through abstract scores. The versions named are Q4_K_M, UD-Q4_K_XL, UD-Q5_K_XL, UD-Q6_K_XL, and UD-Q8_K_XL. The main question is how much speed and someone should give up to run a larger quantized model.
The target machine is a consumer desktop with two GPUs and 32GB of total VRAM. That setup is treated as a realistic off-the-shelf option for work on a reasonable budget. The preferred test setup is llama.cpp on Ubuntu.
The main use cases are coding and more complex processing tasks, with openness to testing other practical workloads too.
Key points
- Several quantized versions are being considered for comparison.
- The key tradeoff is larger model quality versus speed and .
- The target hardware is a consumer desktop with two GPUs and 32GB total VRAM.
- The preferred runtime is llama.cpp on Ubuntu.
- Coding and complex processing tasks are the main practical test cases.