What a Voice Agent Actually Needs

Five systems have to cooperate before a phone call with an AI on the other end sounds like anything at all. Speech to text turns the caller's audio into words. A model decides what to say. Text to speech turns that decision back into audio. A telephony leg carries the whole thing across the phone network. And turn-taking logic answers the hardest question in the stack, which is whether the caller has finished a sentence or is just breathing.

Vendors sell those five in two shapes. You either buy the pieces separately and wire them together, or you buy one speech-to-speech model that transcribes, reasons and synthesises in a single pass. Both shapes still need a phone number. Both shapes still need somebody to tune turn-taking. Every price below comes from a vendor's own published page, read on 31 July 2026.

The three model components, and the two units they are sold in

A cascaded pipeline prices cleanly per minute of audio and per thousand characters of speech. A speech-to-speech model does not always do that, which makes the two architectures awkward to compare on a spreadsheet.

Look at the last two rows together. xAI's pricing page gives you a per-minute number and an hourly equivalent, so you can multiply by expected call minutes and stop. OpenAI's page bills the same job in audio tokens, and audio tokens per minute of speech depend on how much your agent talks. Those two lines are not comparable until you have measured your own traffic.

One more line item hides inside the speech-to-speech row. xAI charges $0.004 for text input on top of the audio rate, which is where your system prompt, your tool results and your injected CRM context land.

Turn-taking is a component, not a checkbox

LiveKit's turn-detection page defines the job plainly: "Turn detection is the process of determining when a user begins or ends their 'turn' in a conversation." It then explains why the naive version fails, since "Most turn detection techniques rely on voice activity detection (VAD) to detect periods of silence in user input." Silence is not the same as finishing a thought, which is why a caller reciting a card number gets cut off after the fourth digit.

LiveKit documents five modes: a turn detector model as the default, the built-in detection inside realtime models, turn_detection="vad" for "minimal latency, or support for a spoken language the turn detector model doesn't cover," turn_detection="stt" to lean on an STT provider's endpointing, and turn_detection="manual" for "push-to-talk or fully explicit control over turn boundaries."

The Grok Speech to Speech API exposes the same job as four session parameters, with defaults published for three of them:

• turn_detection.type: "server_vad" for automatic detection, null for manual text turns.
• turn_detection.threshold: "VAD activation threshold (0.1-0.9). Higher values require louder audio to trigger. Default: 0.85."
• turn_detection.silence_duration_ms: "How long the user must be silent (in ms) before the server ends the turn (0-10000)."
• turn_detection.prefix_padding_ms: audio included before detected speech start, "Default: 333."

That 0.85 default matters more than it looks. It is near the top of the allowed range, which means a quiet caller on a bad handset can fail to trigger the agent at all.

Interruption handling is a separate set of dials again. LiveKit ships min_duration, min_words, false_interruption_timeout, resume_false_interruption and discard_audio_if_uninterruptible, plus two modes, "adaptive" and "vad". Adaptive "is the default mode for agents deployed to LiveKit Cloud." Grok's equivalent advice is one line: "Enable server_vad for automatic, natural barge-in."

The phone leg costs almost nothing and constrains almost everything

Twilio's US voice pricing puts a local number at $1.15 / mo, inbound local calls at $0.0085 / min and outbound local at $0.0140 / min. Toll-free runs $2.15 / mo with inbound at $0.0220 / min. Against a model that charges $0.08 per minute, the carrier is rounding error.

What the phone leg does dictate is audio format. The public switched telephone network runs 8 kHz narrowband G.711, and every voice platform has to say what it does about that. xAI's session parameters accept audio/pcmu (G.711 mu-law, 8000 Hz), audio/pcma (G.711 A-law), audio/opus (24000 Hz) and audio/pcm at seven configurable sample rates. ElevenLabs' SIP trunking page states the requirement as a condition: "Your system must support either G711 or G722 audio codecs or be capable of resampling audio on your end."

The practical consequence is that the studio-grade voice you auditioned in a browser is not the voice your caller hears. It gets resampled down to telephone bandwidth on the way out.

What a bundled per-minute price leaves out

Managed platforms quote one number and hide the stack, which is the point of buying one. Read what the number covers. ElevenLabs' agents pricing reads: "Additional call minutes cost $0.08 per minute (burst pricing $0.16 per minute, double the standard rate, when you exceed your concurrency limit)." The same page notes that LLM and telephony are billed separately based on usage. So $0.08 buys the platform, not the call.

Concurrency is the other buried constraint. The published tiers run from Free at 15 included minutes and 4 concurrent calls, through Creator at $22 with 275 minutes and 10 concurrent, to Business at $990 with 12,375 minutes and 40 concurrent. Exceed the concurrency ceiling and the per-minute rate doubles rather than the call failing, which is a kinder failure mode than most and an easy one to miss on a bill.

Version pinning belongs in the same paragraph. xAI's model table says grok-voice-latest is an "Alias for grok-voice-think-fast-1.0" and "Updates to grok-voice-think-fast-2.0 on August 5, 2026." Those two models are priced at $0.05 and $0.08 per minute. An agent left pointing at the alias gets a 60% per-minute increase on a published date, without anyone touching the code. The docs say to "Pin to a versioned model name in production for stability," and the billing argument for that is as strong as the behavioural one.

The measurement none of them print

Not one of these pages publishes an end-to-end response time. xAI's speech-to-speech documentation has a Best Practices section, and every item in it is advice rather than a figure: start the WebSocket "as early as possible," "Do not wait for the WebSocket open event before starting to collect microphone samples," "Match input/output format (24 kHz PCM) to avoid resampling." Sound engineering, zero numbers. Deepgram and Twilio price by the minute and say nothing about time to first byte anywhere on their pricing pages.

So the latency budget you plan against is one you have to assemble yourself from your own pipeline. LiveKit is the only vendor here that publishes the arithmetic for doing it, and that is a measurement problem rather than a shopping problem.

If you would rather skip the assembly entirely, the no-code route through xAI's console is covered in part 4, and the voice-cloning options that feed the text-to-speech slot are in part 2 and part 3. Managed platforms such as Vapi, Retell AI and Bland AI collapse the same five components behind a single console, and Grok Voice Agent Builder and ElevenLabs do it behind their own. The five components do not go away. They just stop being separately itemised.