A huge open AI model ran on a 25 GB RAM laptop
GLM-5.2, a 744-billion-parameter open AI model, was run on a laptop with 12 CPU cores and 25 GB of RAM. The model uses a MoE design, so only selected parts need to be loaded for each request instead of the whole model at once. A small pure-C engine called colibrì kept the dense parts in memory, using about 10 GB, and loaded the needed expert parts from disk when required.
The work involved , streaming, MTP tuning, and help from . The result was real chat output, including Italian conversation, on modest hardware. The speed is still extremely slow at about 0.05 to 0.1 on a cold run.
The project is early and built by one person, with hopes that people using faster NVMe storage or more RAM can test it and share results.
Key points
- GLM-5.2 with 744 billion parameters ran on a 25 GB RAM laptop.
- colibrì keeps the dense parts in memory and streams expert parts from disk.
- The current speed is only about 0.05 to 0.1 .
- and streaming made the laptop run possible.
- Faster NVMe storage or more RAM may improve results.