One local AI server may replace a messy three-machine setup
A small business is currently running its AI work across three weak machines: a with 16 gigabytes of memory, a Dell desktop with 12 gigabytes of memory, and a mini PC with 8 gigabytes of memory. These machines are connected with SSH tunnels, but there is no shared GPU setup, so the system depends on workarounds rather than a clean design. The goal is to replace this with one on-site machine for local and RAG over private business data.
Light fine-tuning may come later. Three hardware paths are under consideration. The has 96 gigabytes of VRAM and gives the most room for .
The RTX PRO 5000 Blackwell has 48 gigabytes of VRAM, costs less, and may be enough for the real workload. Two RTX 5090 cards offer more raw compute on paper, but they add complexity because models may need to be split across cards and the system has more parts to manage. The main priorities are stability, , enough VRAM to avoid constant model swapping, power and heat that can be handled once, and a setup that does not need daily care.
Key points
- The current setup spreads AI work across a , a Dell desktop, and a mini PC.
- The new goal is one on-site machine for local and RAG using private business data.
- The offers the most flexibility with 96 gigabytes of VRAM.
- The RTX PRO 5000 Blackwell may be the more practical option if 48 gigabytes of VRAM is enough.
- Two RTX 5090 cards may be faster on paper but add model-splitting and maintenance complexity.