mlx-optiq keeps MLX models more accurate at the same small size
mlx-optiq is a tool for running with MLX on . Instead of shrinking every layer to the same 4-bit level, it checks how sensitive each layer is and gives more bits to the layers that need them most. Sensitive layers can use 8-bit, while the rest stay at 4-bit, keeping the average model size about the same.
The resulting models load with the normal `mlx_lm.load(...)` call, so they do not need a special runtime. At the same on-disk size, they keep more accuracy than uniform 4-bit models, with the biggest gains on smaller models. On Google’s Gemma-4 weights without QAT, the result was described as nearly lossless, and it also improved results when combined with QAT at the same bit level.
E2B scored 54.2 versus 52.1 for uniform 4-bit, with 58.6% on GSM8K. E4B scored 65.8 versus 64.6 for uniform 4-bit, with 80.0% on GSM8K.
Key points
- mlx-optiq assigns different bit levels to different model layers instead of using uniform 4-bit everywhere.
- Sensitive layers can use 8-bit while less sensitive layers stay at 4-bit, keeping the average size similar.
- The models load through the standard `mlx_lm.load(...)` path and do not need a special runtime.
- The reported gains are strongest on smaller models at the same on-disk size.
- It also includes a local web lab and an OpenAI/ server for testing local model setups.