A 448GB VRAM home setup runs MiniMax M3
MiniMax M3 was run on a very large personal machine with 448GB of VRAM. The setup used two 96GB RTX Pro 6000 Max-Q cards, eight 24GB RTX 3090 cards, two 32GB RTX 5090 cards, three power supplies, 128GB of DDR5 memory, and a 9960x processor.
The model was reduced with AWQ-INT4 and served with vLLM, using PP and TP groups to split the work across many s. Speed was about 30 for one single stream and about 960 when processing a batch.
The setup can provide a 1 million token context for one user, but the goal is to support four users at once. The best way to place and share that across users is still unresolved.
Key points
- The machine has 448GB of VRAM spread across 12 s.
- AWQ-INT4 was used to shrink the model and reduce .
- vLLM, PP, and TP were used to divide model work across the cards.
- Single-stream speed was about 30 , while batch speed reached about 960 .
- A 1 million token context works for one user, but four-user concurrency is still being worked out.