If full-duplex speech-to-speech is the most natural design, why do most production agents still run on cascades? Because what makes it feel human makes it hard to govern, and because a cascade has structural advantages that a single model gives up.
The first is that a cascade's reply can be intercepted. The LLM finishes a text reply before any audio exists, and that text can be checked by a policy model, redacted for PII, rewritten or blocked before the TTS speaks a word. A speech-to-speech model decodes audio tokens directly, so there is no reply to intercept. By the time a transcript of its output exists, the caller has already heard it, and full duplex removes even the turn boundary where most checks sit. That matters most for required wording. US debt collectors must disclose that a call is from a debt collector [53], and the EU AI Act requires telling people they are interacting with an AI [54]. A cascade can force exact words through the TTS; a speech-to-speech model says what it generates. In our MIVAS harness, text handed to a GPT-Live session as commentary is paraphrased rather than read verbatim.
The second is that a cascade can listen more than once. The same caller audio can go through a fast streaming pass for turn-taking and a slower, more accurate pass for the words that matter, such as a policy number or a medication name. Outputs from several recognizers can be combined, or a final transcript corrected against context before the LLM acts on it, which pushes word error rate below what any single pass achieves. A speech-to-speech model hears the audio once, in one forward pass, and whatever it misheard becomes part of its reasoning.
The third is observability and control. A cascade leaves a transcript, a prompt and a TTS input at every hop, so a failed call can be traced to the stage that failed, and each stage can be swapped independently (Table 3). A speech-to-speech model is one opaque step from audio to audio, and its voice, reasoning and turn-taking come from a single vendor as one unit.
The fourth is iteration speed. STT, LLM and TTS models each optimize a narrow set of metrics, such as word error rate, reasoning accuracy or voice naturalness, so each can improve quickly. A speech-to-speech model has to improve hearing, reasoning, speaking and turn-taking together, and a regression in one holds back a release of the rest. The components also have far more economic activity behind them, because each one serves markets beyond voice agents: TTS narrates social video and audiobooks, STT powers medical dictation, captioning and meeting notes, and LLMs serve nearly every text workload. That spending funds faster progress, and a cascade inherits each component's improvement the day it ships.
Still, speech-to-speech is how people talk. They hear tone, backchannel while the other person speaks and start answering before a sentence ends, and removing the hops between recognition, reasoning and synthesis removes their latency. The likely near-term shape is GPT-Live-1's: a speech-to-speech model on the audio path for naturalness, with a text model behind it for reasoning, tools and the checks enterprises need.