A tiny CPU-only C engine for running small Qwen 3 models
This is a simple that runs small Qwen 3 models up to 4B size using only a computer’s CPU. It was written from scratch in pure C instead of being built by connecting Python libraries. It needs very few outside pieces: the standard C library, the math library, cJSON, and optionally OpenMP for parallel running.
It can load Hugging Face safetensors files directly and apply while the model runs, which makes the model lighter to handle. It also includes KV caching, so a chat session can reuse some earlier work instead of recalculating everything. A built-in chat interface is included, but the engine is very slow.
The main value is that the code is readable and useful for learning how run under the hood.
Key points
- It runs Qwen 3 models up to 4B size on CPU only.
- It is written in pure C and keeps very small.
- It reads safetensors files directly from Hugging Face model files.
- It uses to make model data lighter during running.
- It supports KV caching, but the engine is described as very slow.