Apple Silicon Macs can now convert EXL3 models
Macs can now run, use for inference, and convert EXL3 models. EXL3 is a method that makes smaller so they can run on consumer machines more easily. Until now, high-quality EXL3 was mostly tied to CUDA and Nvidia RTX cards, often needing 96GB to 128GB of on expensive specialist hardware.
macOS and now offer another path, especially on Macs with 64GB or more memory. MiniCPM5 and Qwen3.6-27B were tested with this approach. MiniCPM5 reached a mean KLD similar to a model converted on an RTX card, while Qwen3.6-27B was only slightly behind.
EXL3 is described as having a better quality-to-size tradeoff than MLX by about half a bit per weight in general. The PonyExl3 tool is available on GitHub under the .
Key points
- Macs can now run, do inference with, and convert EXL3 models.
- This was previously mostly tied to CUDA and high-memory Nvidia RTX cards.
- MiniCPM5 and Qwen3.6-27B were tested with the new approach.
- MiniCPM5 conversion quality was reported as similar to an RTX-card conversion.
- PonyExl3 is available on GitHub with an .