antirez's DS4 model impresses even at 2-bit quantization for local use
A user running A3B as their main turns to antirez's DS4 flash as a backup "big brain" whenever the main model gets stuck on a hard problem or needs to plan something out. They report being impressed with how capable DS4 is even at a 2-bit level.
Their hardware is a Nimo box with 128GB of memory, running the kyuz0 software stack with ROCm. They are asking whether anyone has tried the unsloth GGUF build of DS4 at a similar 2-bit and compared its .
Key points
- antirez's DS4 flash is being run locally at 2-bit
- Used as a backup 'big brain' when the main model ( A3B) gets stuck
- Hardware: Nimo box, 128GB memory, kyuz0 stack with ROCm
- Asking the community for a comparison against the unsloth GGUF build