Trying out Bonsai 8B, a sub-2-bit ternary AI model, on real hardware
PrismML released the Bonsai family of extremely AI models, where each weight is stored using only three values (-1, 0, +1, called ternary) or two values (-1, +1, called binary), plus one shared scaling number per group of 128 weights. average about 1.7 bits each, and binary about 1.1 bits — genuinely under 2 bits per weight, unlike typical '2-bit' GGUF models, which actually average closer to 2.8 bits once higher-precision parts are included.
Unlike BitNet, which trains a 1.58-bit model from scratch, Bonsai takes an already-trained model (Qwen3) and converts it into this ultra-compact format using a proprietary technique, so an existing model's carry over. PrismML's newest flagship is a 27B (27 billion parameter) model, but the version tested here is the smaller 8B release, Ternary-Bonsai-8B.
Despite its 8B size, it runs using only about 2GB of memory, though the format is new enough that it needs specific tooling support. PrismML itself notes these early 1.7B-8B releases were not built to handle complex reasoning or reliable tool use.
Key points
- Bonsai stores weights as -1/0/+1 (ternary, ~1.7 bits) or -1/+1 (binary, ~1.1 bits) per weight
- Typical '2-bit' GGUF actually average ~2.8 bits, making Bonsai genuinely smaller
- Unlike BitNet's from-scratch training, Bonsai converts an existing Qwen3 model into this compact format
- The 8B model uses only about 2GB of memory but needs new tooling support since the format is unfamiliar
- Early 1.7B-8B releases were not designed for complex reasoning or reliable tool use