Voice-to-voice latency

What it is

Voice-to-voice latency is measured from the end of the caller's speech to the first audio of the agent's reply. It includes end-of-turn detection, the final speech-to-text result, the language model's first tokens, any tool call the turn needs, text-to-speech start-up and the network path back to the phone. It is the sum of the whole pipeline, which is why component metrics such as time to first audio understate it.

A rule of thumb repeated across vendor and buyer guides puts the comfortable range under roughly 800 milliseconds and the "feels like an IVR" threshold above roughly 1.2 seconds. Treat those as orientation, not as a standard; the source is a vendor blog, and human tolerance varies with context and culture.

Why it matters when buying

Latency is the property vendors most often demo well and ship badly. Demos run on greetings and canned answers; production runs on turns that look up an account or check a calendar, and those turns are slower. Percentiles matter more than averages: a median of 700 milliseconds with a 90th percentile of 2.5 seconds is a conversation that stalls every few turns.

What to ask

Ask for median and 90th percentile, measured from end of caller speech to first agent audio, over a real phone call, on tool-backed turns reported separately. Ask what the agent does during a slow tool call (a filler phrase, a hold tone, silence). Then measure it yourself from a call recording; the methodology page shows how.