MiniMax M3 helper model now runs faster in llama.cpp
A GGUF version of the MiniMax M3 is now available and can run with llama.cpp. The model works as a smaller helper that suggests likely next tokens before the main model finishes, which can make text generation faster. The repository includes steps for merging the needed llama.cpp change and running the model.
In one test using two RTX 3090 s and 128GB of , the UD-Q2_K_XL quantized model improved from 2.3 to 5 . The speed gain came from using the --fit option and keeping the helper model in memory instead of .
Key points
- The MiniMax M3 EAGLE helper model has been converted to GGUF.
- It can run with llama.cpp after applying the needed code change.
- The test used two RTX 3090 s and 128GB of .
- With UD-Q2_K_XL , speed rose from 2.3 to 5 .
- Keeping the helper model in memory was important for the speed gain.