Running local AI workloads on two NVIDIA DGX Spark units
Two units were used for real inference workloads in both testing and production settings. The setup was chosen because it offered the best balance of speed, VRAM, cost, maintenance, and ease of use for the stated needs.
Those needs include testing open source , running in production, and fine-tuning models for custom client or workflow tasks. Other options considered were a server with two to four RTX 6000 cards and an EPYC processor, and a Mac Studio M3 Ultra with 256GB or 512GB of memory.
Models used include , 3 Super, Qwen 3.5 122B, Qwen 3.6 27B, plus smaller embedding, vision, and image models. Privacy is named as one reason to prefer self-hosting over cloud services, but the captured content does not include the rest of that explanation.
Key points
- Two DGX Spark units were used for real testing and production inference workloads.
- The main buying criteria were speed, VRAM, cost, maintenance, and ease of use.
- The setup supports open source model testing, local production use, and fine-tuning.
- Alternatives included RTX 6000 server hardware and a high-memory Mac Studio M3 Ultra.
- The workload included Deepseek, Nemotron, Qwen, embedding, vision, and image models.