audio.cpp makes 10 hours of speech in 3 minutes on an RTX 5090
audio.cpp 0.3 adds five speech-generation models: Supertonic 3, MOSS-TTS-Local, MOSS-TTS-Nano, IndexTTS2, and Irodori-TTS. With CUDA on an RTX 5090, Supertonic 3 produced speech at more than 200 times playback speed; on a CPU, it exceeded six times playback speed. In , the first audio arrived in about 47 milliseconds.
A test using The Adventures of Sherlock Holmes produced roughly 10 hours of audio in about three minutes on one RTX 5090. The official version uses ONNX, while audio.cpp rebuilt the generation process around the model's weights. This let the GPU handle more of the work instead of sending some back to the CPU, making the C++ version much faster than Python with CUDA; CPU was similar or slightly faster.
The other newly supported models ranged from about the same speed to more than twice as fast, depending on the test. In a long IndexTTS2 test containing 6,000 characters of text and 2,400 characters of emotion instructions, C++ was 5.65 times faster than Python, and the release also added GGUF support.
Key points
- Version 0.3 adds five speech-generation models to audio.cpp.
- Supertonic 3 generated about 10 hours of speech in roughly three minutes on an RTX 5090.
- produced its first audio in about 47 milliseconds.
- C++ ran 5.65 times faster than Python in the long IndexTTS2 test.
- GGUF support adds another option for storing and running the models efficiently.