Colibrì runs GLM-5.2 on a slow personal computer
Colibrì is a one-person C project for running GLM-5.2, a 744-billion-parameter AI model, on a consumer computer. It avoids loading the whole model into memory by keeping the dense core in RAM and reading the selected experts from disk only when needed. The int4 model takes about 370GB on disk, while chat uses roughly 20GB of memory under the tested setup.
The result is not fast: the developer’s machine gets about 0.05 to 0.1 when the cache is cold, but it can still talk to a huge model without a GPU. MTP can speed things up by drafting possible next tokens, but it may hurt before the cache warms up because it causes extra disk reads. Setup involves building the C engine and either downloading a pre-converted model or converting the FP8 model shard by shard into int4.
Faster machines should perform better, but the accuracy loss from int4 has not been fully measured yet.
Key points
- Colibrì is a C engine for running the 744-billion-parameter GLM-5.2 model on consumer hardware.
- It keeps part of the model in memory and streams selected experts from disk as needed.
- The int4 model is about 370GB, and chat can stay around 20GB of memory in the tested setup.
- Cold is extremely slow at about 0.05 to 0.1 .
- Accuracy after int4 still needs better testing on faster machines.