Working paper / 2026

MISTT

Multi-condition Impairment Speech-to-Text Bench

Abstract

Multi-condition Impairment Speech-to-Text Bench (MISTT) measures the speech-to-text models that voice agents depend on. Every model hears the same 2,999 frozen clips of voice-agent calls, doctor–patient consultations and five-language parliamentary speech, plus 400 degraded copies of 100 of them: phone lines, low-bitrate codecs, packet loss, background noise and an intelligible background talker. Streaming models are timed the way an agent waits on them, from the forced end of speech to the final transcript. Accuracy carries bootstrap confidence intervals, and the degraded copies are paired with their clean originals, so robustness is measured on the same words.

Leaderboard ↗Read the white paper

v0 · October 2026

← All models

Gemini 3.5 Transcribe Live

Google · Streaming · #7 of 36 streaming by English WER · google-gemini-3-5-transcribe-live-stream

English WER

4.08%

95% CI 3.73% to 4.41%

TTFS p50 / p95

348 ms

p95 504 ms

$ / 1k audio min

$9.88

token-priced estimate

Δ Severe

+6.8 pts

2.24% → 9.05% on 100 clips

Accuracy by dataset

English index	4.08%	3.73% – 4.41%
pipecat (voice-agent turns)	1.55%	1.34% – 1.77%
primock57 (doctor–patient)	6.60%	5.89% – 7.30%
Multilingual (de/fr/es/it pooled)	12.59%	11.60% – 13.67%
VoxPopuli English	7.40%	6.38% – 8.43%
VoxPopuli German	11.76%	10.45% – 13.11%
VoxPopuli French	11.40%	9.61% – 13.27%
VoxPopuli Spanish	8.19%	6.73% – 9.90%
VoxPopuli Italian	18.93%	16.38% – 21.48%

English substitutions 1.66%, deletions 1.76%, insertions 0.65% · raw (un-normalized) English WER 16.87% · hallucination 2.70% of clips · empty 0.07% · VoxPopuli CER 8.62%

Streaming latency

	p50	p90	p95	p99	max
TTFS	348 ms	469 ms	504 ms	586 ms	2.32 s
TTFT	831 ms	1.29 s	1.53 s	2.10 s	6.59 s
TTFB	51 ms	54 ms	55 ms	65 ms	6.26 s
Connection setup	174 ms	191 ms	201 ms	356 ms	1.39 s

English WER@500 ms 4.33% (final 4.08%) · finals before t0 0.0% · fallback finals 0.1% · partial words later rewritten 11.9%

Robustness · same 100 clips

Mix	WER	Δ pts (95% CI)	Worse	Rank
Clean	2.24%	—	—	#9
Light	3.21%	+1.0 (+0.2 to +1.8)	16/100	#29
Moderate	6.33%	+4.1 (+2.7 to +5.7)	37/100	#20
Severe	9.05%	+6.8 (+3.4 to +10.8)	34/100	#40
Crosstalk	8.57%	+6.3 (+3.2 to +9.9)	39/100	#14

Bleed 0.89% of background-talker words · ranks are out of 81 on the subset.

Configuration and pricing

Languages: en, de, fr, es, it
Finalize: activityEnd; ack = the turn's inputTranscription after it
Config notes: Manual activity detection (automaticActivityDetection disabled): activityStart at open, activityEnd at speech end.
Rate: $0.00911/min pay-as-you-go · billed on audio time
Effective: $9.88 per 1k audio min on this run · clean run cost $4.18
Pricing notes: Live API, standard tier: audio in $3.50/1M tok, output $21.00/1M. Audio 25 tok/s -> $0.005250/min (billed on audio sent, incl. our 1 s tail); output 184 tok/min (same transcripts, measured on the batch probe; Live sends no usageMetadata) -> $0.003860. Effective $0.009110/min; Google's blended estimate $0.009/min, AA $9/1k.
Source: https://ai.google.dev/gemini-api/docs/pricing · checked 2026-10-19
Clip outcomes: ok 2,999