USB4 experiment links mini PCs for cheaper local AI runs
An module makes normal USB4 and ports on mini PCs appear like devices. The goal is to let AI tools such as vLLM and RCCL split inference or training across several small computers instead of relying on one expensive machine.
In a test with two 128GB Strix Halo mini PCs, the setup reached about 95Gb/s of two-way raw RDMA bandwidth and about 7 microseconds of one-way delay. It ran MiniMax-M2.7 inference split across two machines because the model did not fit on one box.
A Gemma 3 27B LoRA FSDP training step dropped from 1359 seconds over Ethernet to 126 seconds over four HCA USB4 RDMA links. The code is still research code, much of it was , and it requires loading kernel modules, so it is not .
Key points
- The experiment turns USB4 and ports into an -like connection.
- Two 128GB Strix Halo mini PCs reached about 95Gb/s two-way RDMA bandwidth.
- MiniMax-M2.7 inference was split across two machines because one machine was not enough.
- A Gemma 3 27B LoRA FSDP step improved from 1359 seconds on Ethernet to 126 seconds on USB4 RDMA.
- The code is and not ready for reliable production use.