Cutting first-response delay in a Hindi voicebot
A Hindi voicebot is running into a delay before the first answer starts. The setup uses LiveKit, a phone-call layer, and GPT-4.1 mini. With LiveKit , the LLM TTFT averaged about 1,700 milliseconds.
After moving to and choosing by lowest latency, TTFT is still around 1,000 milliseconds. The target is 400 to 600 milliseconds. Because the service is based in India, direct use of Bedrock or GCP servers in the ap-south region may reduce distance and speed up replies.
There is concern that ’s Mumbai region label may not mean the model is actually handled in Mumbai, because traffic may still pass through and end up in Europe or the United States. On the voice side, Sarvam TTS has about 300 milliseconds of TTFB, and Cartesia is being considered as a possible faster option.
Key points
- The Hindi voicebot uses LiveKit, a phone layer, and GPT-4.1 mini.
- LLM TTFT was about 1,700 milliseconds with LiveKit .
- lowered TTFT to about 1,000 milliseconds, still above the 400 to 600 millisecond goal.
- Direct Bedrock or GCP servers in ap-south may help users in India.
- Sarvam TTS is giving about 300 milliseconds of TTFB, with Cartesia as a possible alternative.