A $2,500 used-parts setup for running large AI models locally
A personal machine for running large AI models can be built for under $2,500 with used server parts and older s. The example setup uses an Epyc and processor for about $460, two P40 24 GB s for about $460 total, and 512 GB of for about $1,000. , storage, and cooling are estimated at up to $580, bringing the full budget to $2,500.
This setup could run compressed Q2, Q3, or Q4 versions of GLM5.2 with cmoe and llama.cpp. It would be slow, but it would be owned and controlled locally instead of rented from a . The s could later be replaced with faster options such as 4080, 3090, or 2080 Ti 22 GB cards.
Other large models such as KimiK2.6, DeepSeek, and MiniMax may also run on this kind of machine. The main limit is speed: it is not a good fit for constantly running AI agents with huge models, but it may work for planning and serious debugging tasks where waiting is acceptable.
Key points
- The proposed used-parts budget is about $2,500 total.
- The example build includes an Epyc board and processor, two P40 24 GB s, and 512 GB of .
- GLM5.2 Q2, Q3, and Q4 versions could run through cmoe and llama.cpp.
- The setup is too slow for heavy with very large models.
- It may be useful for planning, deep analysis, and difficult debugging where speed matters less.
Sources covering this story (4)
- r/LocalLLaMAA $2,500 used-parts setup for running large AI models locally ↗
- r/LocalLLaMAGLM5.2 performance. ↗
- r/LocalLLaMAGLM5.2 on 5x Pro 6000s and a 5090, an expensive journey ↗
- r/LocalLLaMAFollow-up: GLM-5.2 NVFP4 on four DGX Sparks — the MTP mystery is solved, and it's now ~24 tok/s at 128K context ↗