Three cache bugs made Qwen3.5 painfully slow on a Mac Studio
Qwen3.5-122B was used for long-context on an M3 Ultra Mac Studio with 96GB of memory, but follow-up responses took three to five minutes to begin. The model was not the cause; three bugs in the qMLX serving stack prevented earlier work from being reused. A unique message ID changed the on every turn, breaking exact KV cache matching and forcing the entire prompt to be processed again.
Interrupted were not saved, so the stored conversation no longer matched what had actually been shown. A background writer also produced unusable checkpoints that displaced valid ones from storage. After these problems were fixed, prefill fell from minutes to less than a second; in one example, 53,000 tokens were reused from cache and only 33 tokens needed new processing.
The fork and a benchmark script named bench_qmlx.py were released as open source. The changes remain in a separate fork because their hybrid attention s are specific to Qwen and are expected to keep diverging from the general-purpose upstream project.
Key points
- Keep byte-for-byte stable; do not insert a new message ID on every turn.
- Save the delivered portion of an interrupted in .
- Check whether unusable checkpoints are pushing valid cached data out of storage.
- After the fixes, 53,000 cached tokens were reused while only 33 tokens required prefill.
- The Qwen-focused qMLX fork and bench_qmlx.py benchmark script are available as open source.