Rust-based GPU inference aims for safety and lower risk
cuTile Rust is a tool for writing safer code that runs on GPUs. As more GPU code is produced by AI, the hard part shifts from writing code quickly to knowing whether that code can be trusted. cuTile Rust uses Rust’s ownership rules to let the compiler check for memory problems and data races before the code runs.
Work is split into separate output pieces, inputs are shared for reading, and write kernels in a simpler style that the compiler maps onto GPU work blocks. Grout, a Qwen3 , was built with cuTile Rust in collaboration with . In batch-1 decode tests, it reached 171 for Qwen3-4B on an RTX 5090 and 82 for Qwen3-32B on a B200.
That puts it in the same range as vLLM and SGLang for this test. Many Grout kernels still use an unsafe path today, but the goal is to move them to safe versions that generated GPU code can be checked against.
Key points
- cuTile Rust checks GPU code for memory safety and data races at compile time.
- Grout is a Qwen3 built on cuTile Rust with .
- The reported speed is 171 for Qwen3-4B on an RTX 5090.
- The reported speed is 82 for Qwen3-32B on a B200.
- Many kernels still use an unsafe path, so the safe version is not fully complete yet.