Open navigation

Landscape / Nov 2026

The voice AI landscape

An examination of available voice agents, segmented into cascaded versus speech-to-speech and half duplex versus full duplex.

Faraz Siddiqi

Co-founder & CTO, Bluejay

Published

November 15, 2026

Full duplex
Half duplex

Cascade · full duplex

Research only

Though theoretically possible, no hosted cascade listens while it talks yet.

Speech-to-speech · full duplex

Hosted models

Open weights, commercial use

Research only

Platforms

Cascade · half duplex

Voice agent APIs: one endpoint runs the whole call

Platforms

Speech-to-speech · half duplex

Hosted models

Open weights, commercial use

Platforms

Cascade
Speech-to-speech
Bluejay Labs

Figure 1 Voice systems by architecture (columns) and turn-taking (rows), from public documentation as of November 15, 2026. Position within a quadrant carries no meaning. Hover, focus or tap any entry for its documented behavior.

Summary

Bluejay analyzed 81 voice AI systems from 46 vendors, spanning hosted speech-to-speech models, open-weights releases, managed voice agent APIs and orchestration platforms, across two dimensions: architecture, meaning whether the caller's speech is transcribed before a language model sees it, and turn-taking, meaning whether a detector hands the model the turn or the model listens while it speaks. The purpose is to give teams building and buying voice agents a precise way to describe how these systems behave on a call, because labels such as "realtime", "speech-to-speech" and "full duplex" are now used loosely enough that they rarely say.

We define each category precisely, place every system by what its documentation says it does, and walk through each quadrant of the resulting map. We then examine how the term "speech-to-speech" has been stretched to cover cascades and composite architectures, weigh the tradeoffs that keep most enterprises on cascaded systems, and describe three probes a simulated caller can run to tell the categories apart on a live call. The map reflects public documentation, model cards and licenses as of November 15, 2026.

01 / Architecture

What separates a cascade from speech-to-speech?

A cascade chains three systems: speech recognition (STT), a text LLM, and text-to-speech (TTS). Each stage can come from a different vendor, which is why cascades are easy to configure and debug. They also lose information at every boundary. Tone, hesitation and emphasis don't survive transcription, and each hop adds latency.

A speech-to-speech model takes audio in and produces audio out with no text in the middle, so prosody survives end to end. In between sit half-cascades, such as Ultravox, which feed audio straight into the model but emit text for a TTS. We group those with cascades, because the reply is still written before it is spoken.

02 / Turn-taking

What does full duplex actually mean?

A half-duplex system takes turns. Something outside the reply, usually voice activity detection (VAD) or an end-of-turn classifier, decides the caller is done and hands over the turn. If the caller talks over the reply, the reply is cancelled. Barge-in, in other words, is a half-duplex feature.

A full-duplex model processes the caller's audio and its own output continuously. It can backchannel while you explain, keep talking through a cough, or stop because it chose to rather than because a detector fired. Behavioral benchmarks don't settle which kind a system is: Full-Duplex-Bench scores how a system handles pauses, interruptions and backchannels [16], and half-duplex models with good detectors score well on it. Duplex can also describe two different things: the model, and the system a developer actually calls. We classify the system, by its documented behavior.

03 / Cascade, full duplex

Can a cascade be full duplex?

Not in any product we found, but it can be built. In a standard cascade, streaming STT keeps transcribing while the agent talks, so the input side already listens. What keeps the system half duplex is how the LLM is called: a VAD or end-of-turn detector decides when the caller is done, the LLM gets that finished turn, and barge-in is the detector cutting off the TTS. The LLM never sees what the caller says mid-reply, so it can't choose to keep going, backchannel or stop.

DuplexCascade changes the call [70]. It keeps a streaming ASR, a text LLM (Qwen2-7B-Instruct with LoRA) and a streaming TTS, drops the VAD, and feeds the LLM the caller's speech in 0.6-second chunks. On each chunk the LLM emits a token for whether the caller is still speaking, has finished or is interrupting, so the text model itself decides when to talk. On the authors' aggregate of Full-Duplex-Bench turn-taking scores it reaches 0.858, the best of the open systems they evaluated, while scoring 65.4 on VoiceBench against 69.7 for a plain cascade of the same parts. Only a paper and a demo are public, so we list it as research only. No hosted cascade or voice agent API works this way yet.

Multi-stream LLMs, which read new input while they write [71], would let a cascade keep reading the caller while it talks, but that paper only tests text.

04 / Speech-to-speech, full duplex

Full duplex: listening while speaking

Very few systems listen while they talk. GPT-Live-1 takes continuous input audio with no turn detector in the audio path and delegates reasoning and tool use to a backend model [1]. ByteDance's Seeduplex has run in the Doubao app since April 2026 [9]; Volcano Engine documents an API for it, though access may be invite-only [10]. OpenMOSS opened a free beta of MOSS-Live 1.0 in November, which, like GPT-Live-1, pairs a foreground model that listens and speaks with a background model that reasons and calls tools [57]. Phonic-1 waits for the caller's turn to end before a full reply, but its API can have the agent backchannel while the caller is still speaking, and in our own MIVAS recordings it spoke over the caller more often than GPT-Live-1 did [56, 69]. Five open-weights models are licensed for commercial use: Moshi under CC-BY 4.0 [11], PersonaPlex under the NVIDIA Open Model License [12], MiniCPM-o 4.5 and Haru-4B under Apache-2.0 [13, 58], and NVIDIA's Nemotron VoiceChat under OpenMDW-1.1 [68]. Haru-4B, released November 12, is a base model that does not follow instructions yet.

Nemotron VoiceChat is the only open full-duplex model with tool calling, though NVIDIA's model card still describes it as ready for research purposes and recommends at most five tools per session [14]. That leaves a narrow choice for agents that act on a caller's behalf. Of the systems that listen while they talk, GPT-Live-1 and Phonic-1 are the generally available hosted ones with documented tool use; GPT-Live-1 gets there by delegating to a separate backend model rather than calling tools from the audio model itself [1]. For a booking or account agent, full duplex today effectively means one of those two. On the platform side, nine already connect to GPT-Live-1 [23–29, 55, 72]. Retell added it on November 2 for single-prompt agents, and only Vapi labels its integration beta [25]. Bland previewed its own full-duplex model, Tofu-1, to a limited set of enterprises on November 15 [59].

ModelAccessCommercial useTool calling
GPT-Live-1Hosted API✓✓ via backend model
SeeduplexHosted API, access unclear✓?
Phonic-1Hosted API, backchannels opt-in✓✓
MOSS-Live 1.0Hosted API, free beta?✓ via background model
MoshiOpen weights, CC-BY 4.0✓–
PersonaPlex 7BOpen weights, NVIDIA OML✓–
MiniCPM-o 4.5Open weights, Apache-2.0✓–
Haru-4BOpen weights, Apache-2.0 (base model)✓–
Nemotron VoiceChat 11BOpen weights, OpenMDW-1.1✓✓ max 5 tools
Tofu-1Enterprise research preview–?
DessaResearch preview––
SyncLLM, dGSLMPaper / non-instructable weights––

Table 1 Full-duplex models by availability and tool calling.

Platforms with GPT-Live-1LiveKitVapibetaPipecatRetellTwilioTelnyxBandwidthMicrosoft FoundryAgora

05 / Speech-to-speech, half duplex

Speech-to-speech, half duplex: one model, still taking turns

This is where most APIs sold as "realtime" land. One model hears and speaks, so prosody survives and the cascade's hops disappear, but a detector still decides when the caller is done. OpenAI's Realtime models use server_vad or semantic_vad [2]. Gemini Live runs VAD with a silence window of about 800 ms by default and discards its reply on interruption [3]. Grok Voice uses server_vad [5], and Nova 2 Sonic waits 1.5 to 2 seconds of pause depending on its endpointing sensitivity [6].

Hydra shows why the difference between a model and a deployed system matters. smallest.ai calls it a full-duplex model, and it may well be one: the client streams audio continuously, even while Hydra is speaking. But the API wraps it in a server-side VAD and cancels the reply when the caller starts talking [8]. A full-duplex model behind a cancelling gate behaves like a half-duplex system to the caller, the same way LiveKit's PersonaPlex plugin turns an open full-duplex model into a turn-taker by waiting for 500 ms of silence before it replies [23]. We place Hydra by what a caller gets, which puts it here. Qwen3.8-Omni Flash Realtime is documented as supporting full-duplex interaction, but its turns end on server_vad or semantic_vad, so it lands here for the same reason [65]. Google had scheduled two older Gemini Live models to shut down on November 17, then pulled the date for Flash Live 3.1 and moved 2.5 Flash Native Audio to March 31, 2027 [4].

FamilyModelsTurn ends onNote
OpenAI Realtime2.1, 2, 2.1 Miniserver_vad or semantic_vadBarge-in cancels output
Gemini Live3.8, 3.8 Ext, Flash 3.1, 2.5 Native Audio*VAD, about 800 ms of silence*Retires Mar 31, 2027
Grok Voice3.0server_vad2.0 deprecated
Nova Sonic2.5, 2Provider endpointingNova 2 waits 1.5 to 2 s
Qwen Audio Realtime3.0, 3.1 Plus, 3.1 Flashserver_vad or smart_turnDocs say duplex; VAD ends turns
Qwen3.8-OmniFlash Realtimeserver_vad or semantic_vadDocs say full duplex; VAD ends turns
Hydrav1.1Server-side VADFull-duplex model, cancelling gate
Rime PrismEnglishModel end of turn plus silence timeout
Open weightsQwen3-Omni, Step-Audio 2 mini, Kimi-Audio, LFM3-OmniYour own VADApache-2.0, MIT, LFM Open
Sophon-Realtimev1 E2E, open weightsModel decides at each pauseApache-2.0, tool calling

Table 2 How each half-duplex speech-to-speech family ends a turn.

Platforms that run themVapiRetellLiveKit+ Rime PrismPipecatAzure Voice LiveAmazon ConnectGemini Enterprise for CXTwilioBandwidthAgora

06 / Cascade, half duplex

Where do production agents run today?

Most production agents run in the cascade, half-duplex quadrant, which is the largest by far. It splits two ways. Voice agent APIs are a single endpoint that runs the whole loop; platforms are builders where you assemble it. Vendors that offer both appear in both groups. Nearly every entry lets you bring your own LLM, so vendors now compete on the turn detector.

Deepgram folds end-of-turn detection into its Flux recognizer [18] and on October 29 made Think-Live, a model that decides whether to listen, wait or respond, generally available [61]. Zoom's Voice Agent API runs Scribe recognition, an LLM of your choice and Zoom voices behind one WebSocket with automatic turn detection [60]. AssemblyAI, whose Voice Agent API waited for complete values such as phone numbers before ending a turn, deprecated that API on October 26 and now points developers to its STT behind LiveKit or Pipecat [17]. Detectors like these fix the most common half-duplex failure, cutting in at the first pause, without changing the architecture. They still don't let the agent hear the caller while it speaks. Eleven Realtime is sold as speech-to-speech, but ElevenLabs' documentation says it "uses a cascaded architecture" of distinct models [7], so it sits here too.

SystemSTTLLMTTSTurn-taking
Voice agent APIs
ZoomOwnSwapOwnAutomatic turn detection, barge-in
DeepgramOwnSwapSwapFlux end-of-turn, Think-Live; Aura-2 TTS or third party
CartesiaOwnSwapOwnManaged
MistralOwnSwapOwnSemantic, in the STT model
InworldSwapSwapOwnManaged
Eleven RealtimeOwnOwnOwnsemantic_vad
Platforms
VapiSwapSwapSwapSmart endpointing on VAD
RetellSwapSwapSwapOwn turn-taking model
LiveKit AgentsSwapSwapSwapTurn detector on VAD
PipecatSwapSwapSwapSmart Turn
Twilio ConversationRelaySwapSwapSwapManaged by Twilio
ElevenAgentsOwnSwapOwnTurn-taking model
PolyAIRoutedOwnRoutedManaged
BlandOwnOwnOwnManaged

Table 3 Which parts of the loop each cascade locks to the vendor. Own the vendor's own part only, Swap swappable between the vendor's own and third parties, Routed the vendor picks among third parties for you. More cascades, including Azure Voice Live, Ultravox, Fish Audio, Neuphonic, Sarvam, smallest.ai Atoms, Telnyx, Agora, Baseten, Amazon Connect, Gemini Enterprise for CX, Sierra and Synthflow, are in Figure 1 and the appendix.

07 / Definitions

Speech-to-speech as a marketing term

In research, a speech-to-speech model is a single network. It takes raw audio, encodes it natively into audio or multimodal representations, and decodes audio-native tokens as output. Audio goes in and audio comes out, with no auxiliary models handling the encoding or the decoding. Vendors now use the term for any system that takes audio in and returns audio. Three different designs share the label (Figure 2), and only one is a speech-to-speech model.

Which one you get decides whether prosody and hesitation reach the part that writes the reply, how many hops sit on the latency path, and what a benchmark score is measuring.

Three pipelines. A speech-to-speech model maps caller audio straight to agent audio. A speech-to-speech architecture does the same but hands reasoning and tools to a separate text model. A managed cascade passes text from STT to an LLM to TTS behind one endpoint.
Figure 2 Three systems that can all be sold as "speech-to-speech". Only the first is a speech-to-speech model; only the first two keep text off the audio path.

A speech-to-speech model fits that definition: one network from input waveform to output audio tokens, with no separate recognizer in front of it and no separate synthesizer behind it. OpenAI's Realtime models, Gemini Live, Nova Sonic and Moshi fit it. Qwen3-Omni splits reasoning and speaking into a "thinker" and a "talker", but the two are trained together and the talker is conditioned on the thinker's internal representations, so it still counts [42].

A speech-to-speech architecture puts a speech-to-speech model on the audio path and hands reasoning or tool calls to other models. GPT-Live-1 is the clearest case: the live model hears and speaks continuously, and delegates reasoning and tool use to a Responses API model or the developer's own backend [1]. The caller only ever hears the speech-to-speech model, so the system keeps prosody and full-duplex behavior, but the answer to a hard question is composed by a text model. That system contains a speech-to-speech model. It isn't one, and from a research perspective it shouldn't be benchmarked or described as one.

A managed cascade runs STT, an LLM and TTS behind a single endpoint, and some are sold under the speech-to-speech label. Eleven Realtime is announced as "a new class of Speech to Speech models" and documented as "a cascaded architecture: each component is a distinct model" [7]. Protocol compatibility makes the line harder to see from the client: Eleven Realtime and Inworld's Realtime API both speak OpenAI's Realtime protocol [7, 20], so a cascade can sit behind the same events and endpoints as a native model. "Full duplex" has a related problem: it can describe the model or the deployed system, and the two can differ. smallest.ai announces Hydra as a full-duplex model, while its API runs a server-side VAD that cancels the reply when the caller speaks [8]. Baseten's Realtime Voice docs make the same claim, while a VAD and an end-of-turn model end the caller's turn and an interruption cancels the reply [74, 75].

A practical test is to follow the caller's audio. If it reaches a separately trained model whose only output is text before anything is spoken back, the system is a cascade, whatever the product page says. If the only model that touches audio is a speech-to-speech model, the system is speech-to-speech on the audio path, and the open question is whether that model also does the thinking. Figure 1 places each system by the model on its audio path, which puts GPT-Live-1 in the speech-to-speech column and Eleven Realtime in the cascade column.

08 / Probes

How do you tell them apart on a call?

Documentation can be wrong or vague, so placements should be checked against live calls. These three probes produce different behavior from half-duplex and full-duplex systems, and the second also separates silence-based detectors from semantic ones. A simulated caller can run all three.

ProbeWhat the caller doesHalf duplexFull duplex
BackchannelSays "mm-hm" or "right" while the agent is mid-sentence.Treats it as an interruption and drops the sentence.Keeps talking.
Mid-sentence pausePauses for 700 ms while reading out a date of birth.A silence-based detector takes the turn and answers half a question; a semantic detector usually waits.Keeps listening.
CrosstalkStarts a correction while the agent is confirming a detail.Barge-in cancels the audio and the reply restarts.Can yield, acknowledge, or finish its clause first.

Table 4 Probes that separate half-duplex and full-duplex behavior.

09 / Tradeoffs

The tradeoffs of full-duplex speech-to-speech

If full-duplex speech-to-speech is the most natural design, why do most production agents still run on cascades? Because what makes it feel human makes it hard to govern, and because a cascade has structural advantages that a single model gives up.

The first is that a cascade's reply can be intercepted. The LLM finishes a text reply before any audio exists, and that text can be checked by a policy model, redacted for PII, rewritten or blocked before the TTS speaks a word. A speech-to-speech model decodes audio tokens directly, so there is no reply to intercept. By the time a transcript of its output exists, the caller has already heard it, and full duplex removes even the turn boundary where most checks sit. That matters most for required wording. US debt collectors must disclose that a call is from a debt collector [53], and the EU AI Act requires telling people they are interacting with an AI [54]. A cascade can force exact words through the TTS; a speech-to-speech model says what it generates. In our MIVAS harness, text handed to a GPT-Live session as commentary is paraphrased rather than read verbatim.

The second is that a cascade can listen more than once. The same caller audio can go through a fast streaming pass for turn-taking and a slower, more accurate pass for the words that matter, such as a policy number or a medication name. Outputs from several recognizers can be combined, or a final transcript corrected against context before the LLM acts on it, which pushes word error rate below what any single pass achieves. A speech-to-speech model hears the audio once, in one forward pass, and whatever it misheard becomes part of its reasoning.

The third is observability and control. A cascade leaves a transcript, a prompt and a TTS input at every hop, so a failed call can be traced to the stage that failed, and each stage can be swapped independently (Table 3). A speech-to-speech model is one opaque step from audio to audio, and its voice, reasoning and turn-taking come from a single vendor as one unit.

The fourth is iteration speed. STT, LLM and TTS models each optimize a narrow set of metrics, such as word error rate, reasoning accuracy or voice naturalness, so each can improve quickly. A speech-to-speech model has to improve hearing, reasoning, speaking and turn-taking together, and a regression in one holds back a release of the rest. The components also have far more economic activity behind them, because each one serves markets beyond voice agents: TTS narrates social video and audiobooks, STT powers medical dictation, captioning and meeting notes, and LLMs serve nearly every text workload. That spending funds faster progress, and a cascade inherits each component's improvement the day it ships.

Still, speech-to-speech is how people talk. They hear tone, backchannel while the other person speaks and start answering before a sentence ends, and removing the hops between recognition, reasoning and synthesis removes their latency. The likely near-term shape is GPT-Live-1's: a speech-to-speech model on the audio path for naturalness, with a text model behind it for reasoning, tools and the checks enterprises need.

10 / Caveats

Caveats

This map dates quickly. In the three weeks before this snapshot, Retell added GPT-Live-1 [55], Phonic launched Phonic-1 [56], OpenMOSS opened the MOSS-Live beta [57], Higurashi released Haru-4B [58] and Bland previewed Tofu-1 [59]. Over the same stretch AssemblyAI deprecated its Voice Agent API [17], Hume closed its EVI API [62], Deepgram agreed to acquire Daily, Pipecat's maintainer [61], and Google pulled back the retirement of two Gemini Live models [4]. Treat the map as a snapshot of November 15, 2026.

Placements come from documentation, checked against our own MIVAS recordings where we have them. A system that streams audio continuously can still be gated by a server-side detector, so duplex status is hard to verify from outside, and the probes above are the check we'd run. Three November launches stay off the map until their documentation settles it: Fish Audio's Live 3 is sold as full duplex but described alongside an LLM and a cascaded mode [63], Boson AI's Higgs Realtime Duplex is waitlist-only [64], and Tencent's Hy Realtime runs only inside Tencent's apps. Searches in English and Chinese turned up nothing callable from Zhipu, StepFun or Baidu in the window, so regional systems may be missing. We couldn't confirm Seeduplex's API availability, or general-availability status for most GPT-Live integrations.

11 / Method

How we sorted them

  • Each entry is placed by what its documentation says it does, not by what its launch post calls it.
  • A platform appears once for each mode it offers, so Vapi, LiveKit and Pipecat each appear in three quadrants.
  • Models are labeled by availability: hosted API, open weights under a commercial license, or research only (a research license, trial terms, or a paper without usable weights).
  • Products that can no longer be called are left out, including AssemblyAI's Voice Agent API, Hume EVI and Speechmatics Flow. Deprecated products that still work, such as Grok Voice 2.0, and products with an announced retirement date are tagged.
  • Every speech-to-speech model in the half-duplex quadrant was checked twice: against its vendor's API docs for a turn gate and for caller speech cancelling the reply, and, for the thirteen we have run on MIVAS, against stereo recordings of perfect-audio calls, measuring how often the agent spoke while the caller was still speaking. Our cascaded baseline scored zero on that measure and GPT-Live-1 scored 10% of calls; every model kept in the half-duplex quadrant scored at or near zero except Rime Prism, whose overlaps were early replies that its own turn-taking then cut off.
Appendix: every entry, with its documented behavior
SystemGroupVendor or modeDocumented behavior
DuplexCascadeResearch onlySB IntuitionsA streaming ASR, a text LLM (Qwen2-7B-Instruct with LoRA) and a streaming TTS, with no VAD. The LLM reads the caller's speech in 0.6-second chunks and emits a token for still speaking, finished or interrupting, so the text model decides when to talk. Paper and demo only, March 2026.

Table 5 Cascade, full duplex, every entry (n = 1).

SystemGroupVendor or modeDocumented behavior
GPT-Live-1Hosted modelsOpenAITakes continuous input audio with no turn detector in the audio path, so it can listen while it speaks. Reasoning and tools are delegated to a Responses API model or your own backend. Outbound SIP calling opened to all organizations on 2026-11-02.
SeeduplexHosted modelsByteDance · Volcano EngineByteDance's native full-duplex model, live in the Doubao app since 2026-04-09. Volcano Engine documents an API for it, but access may be invite-only and pricing isn't public.
MOSS-Live 1.0Hosted modelsOpenMOSS · betaFree beta API (moss-live-1.0-2026-11-02) taking audio and video in. A foreground model handles listening, turn-taking and speech while a background model handles reasoning and tools, the same split as GPT-Live-1. No weights or license published.
Phonic-1Hosted modelsPhonic · opt-in backchannelsFull replies wait for the caller's turn to end (a turn_taking setting) and caller speech cancels them, but Phonic's API can make the agent backchannel while the caller is still speaking (enable_assistant_backchannel), and tool calls can run during caller speech. In our MIVAS recordings it spoke over the caller in 35% of perfect-audio calls, more often than GPT-Live-1. Launched 2026-10-29.
MoshiOpen weights, commercial useKyutai · CC-BY 4.0Models its own voice and yours as two parallel streams. Weights are CC-BY 4.0, so you can self-host it commercially. No tool calling.
PersonaPlex 7BOpen weights, commercial useNVIDIA · NVIDIA Open Model LicenseBuilt on Moshi, adding role prompts and voice conditioning. The model card says it is ready for commercial use. No tool calling, and its fal hosting is deprecated, so you host it yourself.
MiniCPM-o 4.5Open weights, commercial useOpenBMB · Apache-2.0Listens while it speaks and decides once a second whether to talk. Apache-2.0 weights. No tool calling.
Haru-4BOpen weights, commercial useHigurashi AI · Apache-2.0Released 2026-11-12. Every 133 ms it takes one frame of caller audio and produces one frame of its own audio and text, on a Qwen3.5-4B backbone with no separate recognizer or synthesizer. A pretrained base model: it does not follow instructions or call tools.
Nemotron VoiceChat 11BOpen weights, commercial useNVIDIA · OpenMDW-1.1Full duplex with tool calling, a first for open models. The weights are under OpenMDW-1.1, which permits commercial use, though NVIDIA's model card still describes it as ready for research purposes and recommends at most five tools per session. The hosted 12B on build.nvidia.com runs under trial terms.
DessaResearch onlyBlockAnnounced 2026-10-23: a full-duplex model trained only on audio. Research preview for a small group of safety researchers, with no API or weights.
Tofu-1Research onlyBlandBland's first speech-to-speech model, announced around 2026-11-15 as full duplex for noisy phone calls. Research preview for a limited number of enterprises by request; no API, pricing or published mechanism yet.
SyncLLMResearch onlyMetaA Llama model trained to interleave both speakers on a shared clock. Paper only; we found no released weights.
dGSLMResearch onlyMeta, 2022Two-channel dialogue model trained on raw audio. Weights exist, but it generates both sides of a conversation and can't be instructed like an assistant.
LiveKit AgentsPlatformsGPT-Live-1 pluginGPT-Live plugin on the same AgentSession API. LiveKit also has an experimental, self-hosted PersonaPlex plugin, but it waits for 500 ms of silence before replying, so that integration behaves half duplex. It also ships a Phonic plugin.
VapiPlatformsGPT-Live, betaGPT-Live has its own section in Vapi's docs, marked beta, with a dashboard quickstart.
PipecatPlatformsOpenAILiveLLMServiceShips a GPT-Live service that its docs describe as full-duplex speech-to-speech conversation.
RetellPlatformsGPT-Live-1Added 2026-11-02. GPT-Live-1 speaks to the caller and handles interruptions while the text LLM you pick runs the prompt, tools and knowledge base. Paid accounts, single-prompt agents only.
TwilioPlatformsAgent Connect GPTLiveProviderTwilio Agent Connect connects Voice Media Streams straight to GPT-Live-1.
Telnyx AI AssistantsPlatformsGPT-LiveGPT-Live is available in Telnyx's AI Assistants with your own OpenAI key. Telnyx also lists PersonaPlex and Nemotron VoiceChat as beta options.
Agora ConvoAIPlatformsGPT-Live MLLMAgora added GPT-Live-1 as an MLLM vendor on 2026-09-25. Its docs say GPT-Live doesn't support Agora's turn detection settings, so the model handles turns itself.
BandwidthPlatformsGPT-Live over SIPA carrier, not an agent builder: it connects your calls to GPT-Live over SIP on its Universal Platform, switched on by your account manager.
Microsoft FoundryPlatformshosts GPT-Live-1Azure hosts GPT-Live-1 directly. Microsoft's Voice Live API doesn't offer it yet.

Table 6 Speech-to-speech, full duplex, every entry (n = 22).

SystemGroupVendor or modeDocumented behavior
Grok Voice 3.0Hosted modelsxAINative audio both ways. Replies start only after server_vad ends the caller's turn, or after the client commits the turn itself. Its Full Duplex Bench score measures turn-taking behavior; we found no documented way for it to speak while the caller is speaking, and none in our MIVAS recordings.
Grok Voice 2.0Hosted modelsxAI · deprecatedPrevious-generation Grok voice model, still callable but deprecated; xAI asks pinned sessions to move to 3.0. Same server_vad turn-taking as 3.0.
Gemini 3.8 Live ExtHosted modelsGoogleGemini 3.8 Live with extended thinking. Same activity detection and interruption handling as 3.8 Live.
Gemini 3.8 LiveHosted modelsGoogleNative audio both ways. Automatic activity detection ends the turn (about 800 ms of silence by default; it can be swapped for manual activity signals), and an interruption cancels and discards the ongoing reply. Proactive audio only decides whether to answer, not when to overlap.
Realtime 2.1Hosted modelsOpenAIOne model, audio in and out. Turns end on server_vad (the default) or semantic_vad, or on a manual commit, and caller speech cancels the reply (response.done with status cancelled). OpenAI points full-duplex use to GPT-Live.
Realtime 2Hosted modelsOpenAIThe previous Realtime generation, still current. Same turn detection and barge-in cancellation as 2.1.
Realtime 2.1 MiniHosted modelsOpenAISmaller Realtime model with the same turn detection and barge-in cancellation.
Nova 2.5 SonicHosted modelsAmazonSpeech understanding and generation in one model over a bidirectional stream. Endpointing waits for a pause (about 1.5 to 2 seconds depending on sensitivity), and when the caller speaks it stops generating and switches to listening. Launched 2026-10-05.
Nova 2 SonicHosted modelsAmazon · EOL after Dec 2Endpointing waits for 1.5 to 2 seconds of pause depending on its sensitivity setting, and caller speech stops generation. End of life no sooner than 2026-12-02.
Qwen Audio RealtimeHosted modelsAlibabaEnd-to-end speech model on Alibaba Model Studio (3.0, 3.1 Plus, and 3.1 Flash since 2026-10-27). Its docs call it a duplex model, but turns end on server_vad or smart_turn and new speech interrupts the reply.
Qwen3.8-Omni Flash RealtimeHosted modelsAlibabaOmni model taking text, images, audio and video and returning text and audio. Documented as supporting full-duplex interaction, but turns end on server_vad or semantic_vad, so callers get half-duplex behavior.
Gemini Flash Live 3.1Hosted modelsGoogleSmaller Gemini Live preview model, same VAD. Google had scheduled it to shut down on 2026-11-17, then removed the date; gemini-3.8-live is the named replacement.
Gemini 2.5 Flash Native AudioHosted modelsGoogle · retires Mar 2027Older native-audio model with the same activity detection. One exception: a tool result the developer sends with scheduling INTERRUPT can make it speak over the caller, which is the developer's choice, not the model's. The Gemini API version retires 2027-03-31; the Vertex GA version retires 2026-12-13.
HydraHosted modelssmallest.aismallest.ai lists it as a full-duplex speech-to-speech model, and its docs define that as the caller being able to talk over it. Turns are decided by a server-side VAD and turn detector the developer can't disable, and caller speech cancels the reply, so callers get half-duplex behavior.
PrismHosted modelsRimeRime's docs say Prism decides when the caller has finished and doesn't talk over them. Its semantic end-of-turn can't be disabled and caller speech cancels the reply. In our MIVAS recordings it overlapped the caller in 12.5% of perfect-audio calls; the cases we reviewed were replies started on an early end-of-turn decision and then cut off, not backchannels.
Qwen3-Omni 30BOpen weights, commercial useAlibaba · Apache-2.0Thinker-talker speech-to-speech model with Apache-2.0 weights. No full-duplex claim, so you add your own VAD.
Step-Audio 2 miniOpen weights, commercial useStepFun · Apache-2.0Open speech-to-speech model with Apache-2.0 weights. No full-duplex claim.
Kimi-Audio 7BOpen weights, commercial useMoonshot · MITEnd-to-end speech conversation model with MIT weights. No full-duplex claim.
Sophon-Realtime v1 E2EOpen weights, commercial useAdventists AI · Apache-2.0Released 2026-11-07; its authors call the family full duplex. A 32B thinker writes the reply and a 350M talker speaks it. Its own code holds every reply silent until a VAD finds a pause and the model answers a one-token 'has the user finished?' check, and plays nothing while the user speaks. Apache-2.0, tool calling supported.
LFM3-Omni-2BOpen weights, commercial useLiquid AI · LFM Open LicenseOne end-to-end model that answers in text and speech, with tool calling. It generates on a complete user turn, so you supply the VAD. Commercial use is subject to the LFM Open License's terms.
VapiPlatformsOpenAI RealtimeOpenAI's gpt-realtime models as a native speech-to-speech option. Vapi offers no other vendor's realtime model.
RetellPlatformsgpt-realtime, Grok Voice 3.0Speech-to-speech options are OpenAI's gpt-realtime family (2.1, 2.1 mini, 2, 1.5 and older) and Grok Voice 3.0. No GPT-Live yet.
LiveKit AgentsPlatformsrealtime pluginsRealtime model plugins for OpenAI, Gemini Live, Nova Sonic, Grok Voice, Ultravox, Rime Prism (since 2026-11-05) and Azure.
PipecatPlatformsS2S servicesSpeech-to-speech services for OpenAI Realtime, Gemini Live, Nova Sonic, Ultravox, Grok, Azure Voice Live and others.
Azure Voice LivePlatformsgpt-realtime, azure-realtimeMicrosoft's managed voice API with the gpt-realtime family or its own azure-realtime model and native voices.
Amazon ConnectPlatformsNova SonicPick Speech-to-Speech as the model type to run Nova Sonic in Amazon's contact center.
Gemini Enterprise for CXPlatformsgemini-3.1-flash-liveGoogle's contact-center agent studio, with Gemini Flash Live as a voice-optimized model.
TwilioPlatformsMedia Streams + RealtimeA do-it-yourself pattern from Twilio's tutorials: stream call audio to OpenAI Realtime.
Agora ConvoAIPlatformsMLLM modeRuns OpenAI Realtime, Azure OpenAI Realtime, Gemini Live and xAI Grok in place of its ASR, LLM and TTS. Turns come from Agora's own VAD or the vendor's server or semantic VAD.
BandwidthPlatformsRealtime over SIPConnects calls to OpenAI Realtime over SIP or WebSocket.

Table 7 Speech-to-speech, half duplex, every entry (n = 30).

SystemGroupVendor or modeDocumented behavior
Eleven Realtime APIVoice agent APIsv1 Max, v1 MiniSold as speech-to-speech, but ElevenLabs' Realtime compatibility docs say it uses a cascaded architecture where each component is a distinct model. Turns end on semantic_vad with an eagerness setting. Rolled out to every ElevenAgents workspace on 2026-11-02, with v1 Mini as the default.
Ultravox RealtimeVoice agent APIshalf-cascade · now InworldSpeech goes straight into the LLM with no transcript, but replies come out as text for a TTS. Layered VAD plus a turn model ends the turn. Inworld acquired Ultravox on 2026-09-30; existing agents keep running. Open weights under MIT.
Inworld Realtime APIVoice agent APIspick STT, LLM and voiceConnects STT, the LLM you choose and Inworld TTS in one session over the OpenAI Realtime protocol, with turn detection and interruptions handled for you.
Mistral Voice Agents APIVoice agent APIsVoxtral STT/TTS · any LLMLaunched 2026-10-20 with Voxtral 3. Semantic end of turn inside the STT model, plus barge-in. Can also be self-hosted.
Fish Audio AgentsVoice agent APIsREST and web SDK, betaPublic beta. Speech recognition, turn-taking and speech synthesis are handled for you.
Neuphonic Agents APIVoice agent APIsSTT → GPT-4o → Neuphonic TTSAgents API over WebSocket or LiveKit WebRTC, with GPT-4o and Neuphonic's TTS.
Deepgram Voice Agent APIVoice agent APIsDeepgram STT · any LLM · any TTSSeparate listen, think and speak providers. Listen is Deepgram only; think is OpenAI, Anthropic, Google, Groq, Bedrock or your own; speak is Deepgram (Aura-2, Flux TTS) or ElevenLabs, Cartesia, OpenAI or Polly. Think-Live, a listen, wait or respond decision model, went GA on 2026-10-29.
Cartesia Agents APIVoice agent APIsInk · your LLM · SonicThe same managed agents driven through the API or a WebSocket: Ink STT, an LLM you choose, Sonic TTS.
Zoom Voice Agent APIVoice agent APIsScribe STT · your LLM · Zoom voicesOne WebSocket with listen, think and speak stages: Zoom Scribe recognition, a built-in gemma-4 model or your own OpenAI or Anthropic key, and ten synthesized voices. Automatic turn detection with barge-in. Pipecat added a service for it on 2026-11-12.
Azure Voice LiveVoice agent APIstext model + Azure SpeechThe same API with a text model and Azure Speech voices instead of native audio.
Baseten Realtime VoiceVoice agent APIsSTT → LLM → TTS · private previewOpen models behind one OpenAI Realtime-compatible WebSocket; Baseten's reference agent runs Qwen3-ASR, Gemma 4 and Qwen3-TTS, with optional delegation of tool calls to a stronger model. The docs call it full duplex, but server or semantic VAD plus an end-of-turn model ends the caller's turn, and an interruption cancels the reply. A caller's short acknowledgment doesn't. Private preview since 2026-11-16.
smallest.ai AtomsPlatformsPulse → Electron → Lightningsmallest.ai's own STT, LLM and TTS in one managed agent platform.
Sarvam Voice AgentsPlatformsSaaras → Sarvam LLM → BulbulSarvam's own STT, LLM and TTS for Indian-language telephony agents. No-code builder; no public conversation WebSocket.
VapiPlatformsSTT → LLM → TTSThe default Vapi agent: a transcriber, a model and a voice, with smart endpointing on VAD.
RetellPlatformsLLM pipelineRetell handles telephony, transcription and turn-taking around your LLM and TTS.
BlandPlatformsall in-houseOwns the full stack: telephony, STT, LLM and TTS. No speech-to-speech mode.
ElevenAgentsPlatformsEleven ASR/TTS · any LLMCustom mode lets you pick the LLM, TTS and ASR yourself. Realtime mode runs Eleven Realtime with those settings managed for you. Both are cascades.
LiveKit AgentsPlatformsSTT-LLM-TTS + turn detectorOpen-source framework. The pipeline mode chains any STT, LLM and TTS, with LiveKit's turn detector model on top of VAD.
PipecatPlatformspipeline + Smart TurnOpen-source framework. Smart Turn is an open audio model that predicts end of turn from the waveform. Deepgram announced it is acquiring Daily, Pipecat's maintainer, on 2026-10-29; Pipecat stays BSD-licensed under a new foundation.
Cartesia Managed AgentsPlatformsPlayground builderBuild and test agents in Cartesia's Playground on Ink STT, an LLM you choose and Sonic TTS. It replaces hosted Line, which stops on 2026-12-01.
Twilio ConversationRelayPlatformsTwilio STT/TTS, your LLMTwilio runs the speech; your app gets transcripts over a WebSocket and sends text back. Deepgram STT and ElevenLabs TTS by default. Smart Turn Detection became generally available on 2026-11-13.
Telnyx AI AssistantsPlatformsSTT → LLM → TTSTelnyx's own agent builder, with speaking-plan timers deciding when it replies.
Agora ConvoAIPlatformsASR → LLM → TTSAgora's Conversational AI Engine chains any ASR, LLM and TTS over its real-time network. VAD opens the caller's turn and a semantic end-of-speech model, English and Chinese only, closes it. Agora also maintains the open-source TEN framework, with its own VAD and turn detection models.
Amazon ConnectPlatformsLex or third-party STT + TTSThe classic path: Lex or third-party STT such as Deepgram, with Polly or ElevenLabs voices.
Gemini Enterprise for CXPlatformsgemini-composite-v1A composite of separate listener, thinking and speaker models.
SierraPlatformsVAD, STT, LLM, TTSRuns its own VAD, transcription, LLM and TTS. Sierra says combined voice-to-voice models aren't yet reliable enough for its enterprise work.
PolyAIPlatformsRaven LLM · routed ASR/TTSIts own Raven LLM. PolyAI routes speech recognition and TTS across its own and many third-party providers by language and domain.
SynthflowPlatformsDeepgram STT + own TTSNo-code agent builder on Deepgram (or its own) STT and its own TTS since May 2026. No speech-to-speech option.

Table 8 Cascade, half duplex, every entry (n = 28).

12 / Sources

Sources

  1. [1]OpenAI. GPT-Live-1 model documentation. developers.openai.com/api/docs/models/gpt-live-1
  2. [2]OpenAI. Realtime API: voice activity detection. developers.openai.com/api/docs/guides/realtime-vad
  3. [3]Google. Gemini Live API guide. ai.google.dev/gemini-api/docs/live-guide
  4. [4]Google. Gemini API deprecations. ai.google.dev/gemini-api/docs/deprecations
  5. [5]xAI. Voice agent guide. docs.x.ai/docs/guides/voice/agent
  6. [6]Amazon. Nova 2.5 Sonic model card; Nova 2 Sonic turn-taking. docs.aws.amazon.com/bedrock/latest/userguide/model-card-amazon-nova-2-5-sonic.html
  7. [7]ElevenLabs. OpenAI Realtime API compatibility (Realtime model architecture). elevenlabs.io/docs/eleven-agents/guides/openai-realtime-compatibility
  8. [8]smallest.ai. Hydra: turn detection and barge-in. docs.smallest.ai/waves/documentation/speech-to-speech-hydra/turn-detection-barge-in
  9. [9]IT之家. ByteDance releases full-duplex speech model Seeduplex (9 April 2026). www.ithome.com/0/937/362.htm
  10. [10]Volcano Engine. End-to-end realtime speech (Seeduplex) API documentation. docs.volcengine.com/docs/6561/2549778
  11. [11]Kyutai. Moshi. github.com/kyutai-labs/moshi
  12. [12]NVIDIA. PersonaPlex 7B v1 model card. huggingface.co/nvidia/personaplex-7b-v1
  13. [13]OpenBMB. MiniCPM-o 4.5 model card. huggingface.co/openbmb/MiniCPM-o-4_5
  14. [14]NVIDIA. NemotronLabs VoiceChat model card (NGC). catalog.ngc.nvidia.com/orgs/nim/nvidia/models/nemotron-labs-voicechat/-
  15. [15]Voice AI Space. Block launches Dessa, a full-duplex speech model trained only on audio (23 October 2026). www.voiceaispace.com/articles/block-launches-dessa-a-full-duplex-speech-model-trained-only-on-audio
  16. [16]Full-Duplex-Bench-v3: Benchmarking tool use for full-duplex voice agents under real-world disfluency. arXiv:2604.04847. www.alphaxiv.org/abs/2604.04847
  17. [17]AssemblyAI. Best speech-to-speech voice agent API (notes the Voice Agent API's deprecation on 2026-10-26). www.assemblyai.com/blog/best-speech-to-speech-voice-agent-api
  18. [18]Deepgram. Configure the Voice Agent. developers.deepgram.com/docs/configure-voice-agent
  19. [19]Cartesia. 2026 changelog. docs.cartesia.ai/changelog/2026
  20. [20]Inworld. Realtime API overview. docs.inworld.ai/realtime/overview.md
  21. [21]Inworld. Inworld acquires Ultravox (30 September 2026). inworld.ai/blog/inworld-acquires-ultravox
  22. [22]Mistral AI. Voxtral 3 and the Voice Agents API (20 October 2026). mistral.ai/news/voxtral-3
  23. [23]LiveKit. Realtime models and the GPT-Live plugin. docs.livekit.io/agents/models/realtime/
  24. [24]Pipecat. OpenAI Live speech-to-speech service. docs.pipecat.ai/api-reference/server/services/s2s/openai-live
  25. [25]Vapi. GPT-Live overview. docs.vapi.ai/gpt-live/overview
  26. [26]Twilio. Twilio and OpenAI GPT-Live-1 API resources. www.twilio.com/en-us/blog/developers/twilio-openai-gpt-live-1-api-resources
  27. [27]Telnyx. OpenAI GPT-Live in Voice AI Assistants. telnyx.com/release-notes/openai-gpt-live-voice-ai-assistants
  28. [28]Bandwidth. OpenAI GPT-Live over SIP. dev.bandwidth.com/docs/voice/integrations/openai/live/sip/
  29. [29]Microsoft. GPT-Live in Azure AI Foundry. learn.microsoft.com/en-us/azure/foundry/openai/concepts/gpt-live
  30. [30]Retell AI. Create Retell LLM (speech-to-speech models). docs.retellai.com/api-references/create-retell-llm
  31. [31]Microsoft. Azure Voice Live API. learn.microsoft.com/en-us/azure/ai-services/speech-service/voice-live
  32. [32]Amazon. Amazon Connect with Nova Sonic speech-to-speech. docs.aws.amazon.com/connect/latest/adminguide/nova-sonic-speech-to-speech.html
  33. [33]Google. Gemini Enterprise for CX: agent configuration. docs.cloud.google.com/gemini-enterprise-cx/cx-agent-studio/agent
  34. [34]Bland. Documentation index. docs.bland.ai/llms.txt
  35. [35]ElevenLabs. ElevenAgents realtime models. elevenlabs.io/docs/eleven-agents/customization/realtime-models
  36. [36]Twilio. ConversationRelay. www.twilio.com/docs/voice/twiml/connect/conversationrelay
  37. [37]Telnyx. AI Assistants. developers.telnyx.com/docs/inference/ai-assistants
  38. [38]Sierra. Voice latency. sierra.ai/blog/voice-latency
  39. [39]PolyAI. Architecture. docs.poly.ai/glossary/architecture
  40. [40]Synthflow. Changelog, 13 May 2026. docs.synthflow.ai/changelog/2026/5/13
  41. [41]Alibaba Cloud. Qwen Audio Realtime user guide. www.alibabacloud.com/help/en/model-studio/qwen-audio-realtime-user-guides
  42. [42]Qwen. Qwen3-Omni-30B-A3B-Instruct model card. huggingface.co/Qwen/Qwen3-Omni-30B-A3B-Instruct
  43. [43]StepFun. Step-Audio 2 mini model card. huggingface.co/stepfun-ai/Step-Audio-2-mini
  44. [44]Moonshot AI. Kimi-Audio-7B-Instruct model card. huggingface.co/moonshotai/Kimi-Audio-7B-Instruct
  45. [45]Fixie.AI. Ultravox. github.com/fixie-ai/ultravox
  46. [46]Rime. Prism overview. docs.rime.ai/docs/prism/overview.md
  47. [47]Phonic. Documentation. docs.phonic.co
  48. [48]Hume AI. EVI overview and sunset notice. dev.hume.ai/docs/speech-to-speech-evi/overview
  49. [49]Speechmatics. Flow API discontinued (GitHub issue #203). github.com/speechmatics/speechmatics-js-sdk/issues/203
  50. [50]Fish Audio. Agents overview. docs.fish.audio/agents/overview
  51. [51]Neuphonic. Documentation. docs.neuphonic.com
  52. [52]Sarvam AI. Changelog. docs.sarvam.ai/changelog.md
  53. [53]Consumer Financial Protection Bureau. Regulation F, 12 CFR 1006.18: false, deceptive or misleading representations (required disclosures). www.ecfr.gov/current/title-12/chapter-X/part-1006/subpart-B/section-1006.18
  54. [54]European Union. Artificial Intelligence Act, Article 50: transparency obligations for providers and deployers of certain AI systems. artificialintelligenceact.eu/article/50/
  55. [55]Retell AI. Changelog: GPT-Live-1 voice model (2026-11-02). www.retellai.com/changelog
  56. [56]Phonic. Introducing Phonic-1 (2026-10-29). phonic.ai/blog/phonic-1
  57. [57]OpenMOSS. MOSS-Live documentation. platform.moss.cn/en/docs/models/moss-live/
  58. [58]Higurashi AI. Haru-4B model card. huggingface.co/higurashi-ai/Haru-4B
  59. [59]Bland. Speech-to-speech (Tofu-1). www.bland.ai/speech-to-speech
  60. [60]Zoom. Voice Agent API documentation. developers.zoom.us/docs/ai-services/voice-agent/
  61. [61]Deepgram. Deepgram, Daily and Pipecat (2026-10-29). deepgram.com/blog/daily-pipecat-deepgram
  62. [62]Hume AI. Voice changelog (EVI shutdown, 2026-11-13). dev.hume.ai/voice/changelog
  63. [63]Fish Audio. Introducing Live 3 (2026-11-10). fish.audio/blog/introducing-live-3/
  64. [64]Boson AI. Higgs Realtime Duplex (2026-11-05). www.boson.ai/blog/higgs-realtime-duplex
  65. [65]Alibaba Cloud. Qwen3.8-Omni-Flash-Realtime. alibabacloud.com/help/en/model-studio/qwen3-8-omni-flash-realtime
  66. [66]Adventists AI. Sophon-Realtime v1 model card. huggingface.co/adventists-ai/Sophon-Realtime-v1
  67. [67]Liquid AI. LFM3-Omni-2B model card. huggingface.co/LiquidAI/LFM3-Omni-2B
  68. [68]NVIDIA. NemotronLabs VoiceChat 11B on Hugging Face (OpenMDW-1.1 license). huggingface.co/nvidia/NVIDIA-NemotronLabs-VoiceChat-11B
  69. [69]Phonic. Conversations API reference (turn_taking, min_words_to_interrupt, enable_assistant_backchannel). docs.phonic.ai/api-reference/conversations/conversations
  70. [70]SB Intuitions and the University of Tokyo. DuplexCascade: full-duplex speech-to-speech dialogue with a VAD-free cascaded ASR-LLM-TTS pipeline. arXiv:2603.09180, 2026. arxiv.org/abs/2603.09180
  71. [71]Su, Yang, Li and Geiping. Multi-Stream LLMs: unblocking language models with parallel streams of thoughts, inputs and outputs. arXiv:2605.12460, 2026. arxiv.org/abs/2605.12460
  72. [72]Agora. Conversational AI: OpenAI GPT-Live. docs.agora.io/en/ai/models/mllm/openai-gpt-live
  73. [73]Agora. Conversational AI release notes (v2.6 turn detection and interruption, v2.13 GPT-Live). docs.agora.io/en/ai/release-notes
  74. [74]Baseten. Realtime Voice overview. docs.baseten.co/inference/audio/realtime-voice/overview
  75. [75]Baseten. Realtime Voice API reference (turn_detection, response cancellation). docs.baseten.co/reference/inference-api/predict-endpoints/realtime-voice-api