ARM CPU change makes NVFP4 model runs about 5x faster
A new llama.cpp change makes NVFP4 models run faster on ARM CPUs. The key idea is to turn UE4M3 values into normal numbers by looking them up in a prepared table instead of doing slower repeated decoding work.
This brings the ARM path closer to the x86 path, where a similar already existed. In a CPU test with a Qwen3.5 4B NVFP4 model using 4 threads, processing a 512- rose from 1.89 to 9.97 .
That is about a 5.3x speedup. The change also reuses the existing shared , so the ARM and x86 versions stay more consistent.
Key points
- llama.cpp is adding a faster ARM CPU path for NVFP4 calculations.
- UE4M3 values are decoded through an existing .
- A Qwen3.5 4B NVFP4 test improved from 1.89 to 9.97 .
- The measured prompt-processing speedup is about 5.3x under the shared test setup.
- This can help lower latency and for .