Working paper / 2026
Multi-condition Impairment Speech-to-Text Bench
Abstract
Multi-condition Impairment Speech-to-Text Bench (MISTT) measures the speech-to-text models that voice agents depend on. Every model hears the same 2,999 frozen clips of voice-agent calls, doctor–patient consultations and five-language parliamentary speech, plus 400 degraded copies of 100 of them: phone lines, low-bitrate codecs, packet loss, background noise and an intelligible background talker. Streaming models are timed the way an agent waits on them, from the forced end of speech to the final transcript. Accuracy carries bootstrap confidence intervals, and the degraded copies are paired with their clean originals, so robustness is measured on the same words.
v0 · October 2026
MISTT draws its audio from public corpora chosen to resemble what a voice agent hears: requests to an assistant, clinical conversation, and speech in five languages. Together they provide 2,999 clips, 7.06 hours and 65,106 reference words per model.
Each entry documents what the corpus contains, why it was selected, how MISTT samples and formats it, how it performs across the evaluated models, and how to obtain and cite it. New corpora are added here as the benchmark grows.
Voice-agent requests
Short, natural requests spoken to a voice assistant. The closest public analogue to the turns a voice agent transcribes.
pipecat-ai/stt-benchmark-data · CC BY 4.0
Clinical consultations
Mock primary-care consultations with manual transcripts. Conversational, disfluent and dense with drug names, doses and symptoms.
babylonhealth/primock57 · CC BY 4.0
Multilingual speech
European Parliament speech in English, German, French, Spanish and Italian, from one setting, with official transcripts.
facebook/voxpopuli · CC0 1.0
Perturbation sources are documented in the methodology: background noise from MUSAN (noise/free-sound, CC BY 4.0; 21 non-speech files after screening) and competing talkers from the AMI Meeting Corpus (distant microphone, CC BY 4.0).
