Bluejay / Labs
Open navigation

Working paper / 2026

MIVAS

Multi-Industry Voice Agent Simulation Bench

Abstract

Multi-Industry Voice Agent Simulation Bench (MIVAS) is an indicator of voice AI performance across economic sectors. MIVAS is comprised of tasks and verifiers with granular rewards executed in production-grade multi-agent RL environments across industries where voice AI is being adopted the fastest. Each task places a model harness inside a multi-agent graph with production-length prompts, tools, and isolated state. Deterministic verifiers score tool use, handoffs, and final database state; pass@1 and passk measure single-run capability and repeated-run reliability.

GitHub repository ↗

Clone the repository, pair a model harness with an industry task suite, and reproduce the evaluation environment.

Legal tasks

Plaintiff-firm intake under conflict-before-facts, no legal advice, and attorney-only declines.

72 tasks

C1-E1
C1-E1: Message for Daniel Okonkwo
Legal · Reception and routing · Renee Whitmore
easy
C1-E1-BG
C1-E1-BG: Message for Daniel Okonkwo
Legal · Reception and routing · Renee Whitmore
easy
C1-E1-SIG
C1-E1-SIG: Message for Daniel Okonkwo
Legal · Reception and routing · Renee Whitmore
easy
C1-E2
C1-E2: Demands a person immediately
Legal · Reception and routing · Gordon Fitzhugh
easy
C1-H1
C1-H1: Third party asks about mother's case
Legal · Reception and routing · Eleanor Vance
hard
C1-H2
C1-H2: Message then premises intake
Legal · Reception and routing · Sam Porter
hard
C1-H3
C1-H3: Medical emergency on the line
Legal · Reception and routing · Inez Calderon
hard
C1-H4
C1-H4: Injured relative cannot speak
Legal · Reception and routing · Corinne Halberg
hard
C1-M1
C1-M1: Returning caller message with lookup
Legal · Reception and routing · Dana Whitfield
medium
C1-M2
C1-M2: Identification fails twice
Legal · Reception and routing · Curtis Beaumont
medium
C1-M3
C1-M3: Cut-off number then complete
Legal · Reception and routing · Wendell Marsh
medium
C1-M4
C1-M4: New auto accident screening checks
Legal · Reception and routing · Harriet Voss
medium
C2-E1
C2-E1: Conflict hit, Vertex Logistics
Legal · Conflicts and barred callers · Howard Fairchild
easy
C2-E1-BG
C2-E1-BG: Conflict hit, Vertex Logistics
Legal · Conflicts and barred callers · Howard Fairchild
easy
C2-E1-SIG
C2-E1-SIG: Conflict hit, Vertex Logistics
Legal · Conflicts and barred callers · Howard Fairchild
easy
C2-E2
C2-E2: Caller already represented elsewhere
Legal · Conflicts and barred callers · Marcus Oyelaran
easy
C2-H1
C2-H1: Conflict unclear, Harlow Properties
Legal · Conflicts and barred callers · Bernice Tolliver
hard
C2-H2
C2-H2: Conflict unclear, hospital and surgeon
Legal · Conflicts and barred callers · Gloria Mensah
hard
C2-H3
C2-H3: Insurance adjuster calls
Legal · Conflicts and barred callers · Nathaniel Croft
hard
C2-H4
C2-H4: Other driver reveals after looking like a new client
Legal · Conflicts and barred callers · Wade Killian
hard
C2-M1
C2-M1: Conflict hit, Northgate Insurance
Legal · Conflicts and barred callers · Selma Ridgeway
medium
C2-M2
C2-M2: Refuses to name the other side
Legal · Conflicts and barred callers · Elton Bragg
medium
C2-M3
C2-M3: Says she is firing her lawyer
Legal · Conflicts and barred callers · Marguerite Okafor
medium
C2-M4
C2-M4: Consulted a lawyer but did not hire
Legal · Conflicts and barred callers · Lois Penney
medium
C3-E1
C3-E1: Bankruptcy matter declined
Legal · Eligibility gates · Silas Dunmore
easy
C3-E1-BG
C3-E1-BG: Bankruptcy matter declined
Legal · Eligibility gates · Silas Dunmore
easy
C3-E1-SIG
C3-E1-SIG: Bankruptcy matter declined
Legal · Eligibility gates · Silas Dunmore
easy
C3-E2
C3-E2: Family matter declined
Legal · Eligibility gates · Marguerite Okafor
easy
C3-H1
C3-H1: Caller starts the story before conflict check
Legal · Eligibility gates · Yvonne Pressley
hard
C3-H2
C3-H2: Filing deadline expired
Legal · Eligibility gates · Pauline Hart
hard
C3-H3
C3-H3: Tight deadline but books intake
Legal · Eligibility gates · Nadine Carlisle
hard
C3-H4
C3-H4: Employment in Washington clears screening
Legal · Eligibility gates · Wilhelmina Prado
hard
C3-M1
C3-M1: Medical malpractice in California
Legal · Eligibility gates · Yvonne Pressley
medium
C3-M2
C3-M2: Workers comp in New York
Legal · Eligibility gates · Caroline Delaney
medium
C3-M3
C3-M3: Auto accident in Arizona
Legal · Eligibility gates · Rita Chen
medium
C3-M4
C3-M4: Immigration matter declined
Legal · Eligibility gates · Rafael Osorio
medium
C4-E1
C4-E1: Intake recorded for auto accident
Legal · Intake and documents · Nadine Carlisle
easy
C4-E1-BG
C4-E1-BG: Intake recorded for auto accident
Legal · Intake and documents · Nadine Carlisle
easy
C4-E1-SIG
C4-E1-SIG: Intake recorded for auto accident
Legal · Intake and documents · Nadine Carlisle
easy
C4-E2
C4-E2: Intake carries screened values
Legal · Intake and documents · Martin Iwu
easy
C4-H1
C4-H1: Product injury intake with packet and note
Legal · Intake and documents · Franklin Deshpande
hard
C4-H2
C4-H2: Consumer intake, declines booking
Legal · Intake and documents · Estelle Kowalczyk
hard
C4-H3
C4-H3: Employment intake in Washington with packet and note
Legal · Intake and documents · Clara Whitman
hard
C4-H4
C4-H4: Medical malpractice Florida intake with records
Legal · Intake and documents · Aisha Rahman
hard
C4-M1
C4-M1: Medical records authorization
Legal · Intake and documents · Franklin Deshpande
medium
C4-M2
C4-M2: Witness detail captured as note
Legal · Intake and documents · Franklin Deshpande
medium
C4-M3
C4-M3: Packet by text message
Legal · Intake and documents · Priya Nair
medium
C4-M4
C4-M4: Auto accident intake with packet and records
Legal · Intake and documents · Chester Lang
medium
C5-E1
C5-E1: Books first offered evaluation slot
Legal · Fees and booking · Delphine Mercer
easy
C5-E1-BG
C5-E1-BG: Books first offered evaluation slot
Legal · Fees and booking · Delphine Mercer
easy
C5-E1-SIG
C5-E1-SIG: Books first offered evaluation slot
Legal · Fees and booking · Delphine Mercer
easy
C5-E2
C5-E2: Declines after fee disclosure
Legal · Fees and booking · Percival Ndiaye
easy
C5-H1
C5-H1: Medical malpractice booked in Florida
Legal · Fees and booking · Amina Okoro
hard
C5-H2
C5-H2: Book then cancel and rebook
Legal · Fees and booking · Delphine Mercer
hard
C5-H3
C5-H3: Mumbled agreement is not a yes
Legal · Fees and booking · Lorraine Pike
hard
C5-H4
C5-H4: Premises evaluation booked in Texas
Legal · Fees and booking · Hector Salinas
hard
C5-M1
C5-M1: Contingency fee hold and confirm
Legal · Fees and booking · Delphine Mercer
medium
C5-M2
C5-M2: Workers comp booking in Florida
Legal · Fees and booking · Martin Iwu
medium
C5-M3
C5-M3: Hourly consumer consultation
Legal · Fees and booking · Estelle Kowalczyk
medium
C5-M4
C5-M4: Employment evaluation booked in Washington
Legal · Fees and booking · Marisol Quintana
medium
R-E1
R-E1: Existing client asks for status
Legal · Clients and refusals · Thomas Eckhart
easy
R-E1-BG
R-E1-BG: Existing client asks for status
Legal · Clients and refusals · Thomas Eckhart
easy
R-E1-SIG
R-E1-SIG: Existing client asks for status
Legal · Clients and refusals · Thomas Eckhart
easy
R-E2
R-E2: Settlement offer advice refused
Legal · Clients and refusals · Thomas Eckhart
easy
R-H1
R-H1: Impersonating firm staff
Legal · Clients and refusals · Karen Delacroix
hard
R-H2
R-H2: Asks to delete the recording
Legal · Clients and refusals · Vince Holloway
hard
R-H3
R-H3: Prompt injection then crash intake
Legal · Clients and refusals · Raymond Cho
hard
R-H4
R-H4: Client status then insists on settlement advice
Legal · Clients and refusals · Thomas Eckhart
hard
R-M1
R-M1: Asks what the case is worth
Legal · Clients and refusals · Estelle Kowalczyk
medium
R-M2
R-M2: Is eighty thousand in the right area
Legal · Clients and refusals · Percival Ndiaye
medium
R-M3
R-M3: Claims a matter the firm has no record of
Legal · Clients and refusals · Jordan Lee
medium
R-M4
R-M4: Client status then message for case manager
Legal · Clients and refusals · Thomas Eckhart
medium