Working paper / 2026
Multi-condition Impairment Speech-to-Text Bench
Abstract
Multi-condition Impairment Speech-to-Text Bench (MISTT) measures the speech-to-text models that voice agents depend on. Every model hears the same 2,999 frozen clips of voice-agent calls, doctor–patient consultations and five-language parliamentary speech, plus 400 degraded copies of 100 of them: phone lines, low-bitrate codecs, packet loss, background noise and an intelligible background talker. Streaming models are timed the way an agent waits on them, from the forced end of speech to the final transcript. Accuracy carries bootstrap confidence intervals, and the degraded copies are paired with their clean originals, so robustness is measured on the same words.
v0 · October 2026
81 ranked models from 25 vendors, grouped by vendor. Each page has the exact configuration, pricing basis and every metric with its interval.
Not yet ranked
A model is ranked only when every clip of both sets finished (scored, refused by the provider, or out of its language scope). These 20 did not, so including them would compare them on different clips. They will be added when their runs complete.
| Config | Vendor | Clean clips done | Reason |
|---|---|---|---|
| alebex-stt-realtime-stream | Alebex | 2,993 / 2,999 | last 7 of 2,999 clips blocked by account limits |
| cohere-cohere-transcribe-03-2026-batch | Cohere | 1,009 / 2,999 | trial-tier monthly cap |
| elevenlabs-scribe-v2-realtime-stream | ElevenLabs | 1,105 / 2,999 | account ran out of credit mid-run |
| gradium-default-batch | Gradium | 610 / 2,999 | account ran out of credit mid-run |
| gradium-default-stream | Gradium | 605 / 2,999 | account ran out of credit mid-run |
| inworld-stt-1-batch | Inworld | 2,537 / 2,999 | account ran out of credit mid-run |
| inworld-stt-1-stream | Inworld | 82 / 2,999 | account ran out of credit mid-run |
| mistral-voxtral-small-2507-batch | Mistral | 2,992 / 2,999 | last 7 of 2,999 clips blocked by account limits |
| replicate-canary-qwen-2-5b-batch | Replicate | 2,282 / 2,999 | vendor rate limits |
| replicate-incredibly-fast-whisper-batch | Replicate | 1,186 / 2,999 | vendor rate limits |
| replicate-parakeet-rnnt-1-1b-batch | Replicate | 2,350 / 2,999 | vendor rate limits |
| replicate-whisper-batch | Replicate | 850 / 2,999 | vendor rate limits |
| reson8-resonant-1-batch | Reson8 | 1,657 / 2,999 | vendor rate limits |
| reson8-resonant-1-realtime-stream | Reson8 | 1,213 / 2,999 | vendor rate limits |
| revai-machine-batch | Rev AI | 844 / 2,999 | account ran out of credit mid-run |
| revai-machine-stream | Rev AI | 22 / 2,999 | account ran out of credit mid-run |
| speechmatics-linden-1-stream | Speechmatics | 2,998 / 2,999 | 2,998 of 2,999 clean clips; not run on the degraded set |
| telnyx-deepgram-nova-2-batch | Telnyx | 2,813 / 2,999 | vendor rate limits |
| telnyx-deepgram-nova-3-batch | Telnyx | 2,819 / 2,999 | vendor rate limits |
| telnyx-openai-whisper-large-v3-turbo-batch | Telnyx | 2,932 / 2,999 | vendor rate limits |
