The judges, scored against humans
Not a model leaderboard, a leaderboard of the judges themselves: how well each SLM judge tracks human ratings on our TTS evals. Agreement is the Spearman ρ (rank correlation) between the judge's per-generation score and individual human votes on the same generation/question, treating human votes as ground truth. Each eval's ρ is computed separately and averaged afterward, avoiding pooled-score distortions from eval difficulty; category values are the mean of their evals' within-eval ρ's.
- judges evaluated
- 7
- factors scored
- 8
- agreement metric
- Spearman ρ
- licenses ranked
- Open + closed
judges evaluated
factors scored
agreement metric
licenses ranked
Live standings
SLM Judge model rankings
Select a model to see every metric, filter by open-source or proprietary models, and switch tabs to rank by a different condition.
SLM Judge - Acting / Role-fit
Judge–human agreement (mean within-eval Spearman ρ) on the acting evals, how convincingly the voice adopts a performative role with matching style, timing, and register.
Showing 7 of 7 models
Scoring methodology varies by category. Select a category above to see how its scores are computed.
What we test
Whether a judge can stand in for a human
An automated judge is only useful if it agrees with the people it replaces. We score that agreement directly.
Judges, not models
This board ranks the SLM judges themselves, how closely each one's scores track human ratings, not the text-to-speech models they score.
Human agreement
Agreement is the Spearman rank correlation between a judge's per-generation score and individual human votes on the same clip, with human votes as ground truth.
Per-factor, then averaged
Each eval's agreement is computed separately and averaged, so a judge is scored per expressive factor rather than on a single pooled number.
Methodology
How the scores are computed
Agreement is the Spearman ρ (rank correlation) between a judge's per-generation score and individual human votes on the same generation, treating human votes as ground truth. Higher is better.
Each eval's ρ is computed separately and then averaged, which avoids pooled-score distortions from differing eval difficulty. Each factor's value is the mean of its evals' within-eval ρ's.
This is a different question from the model boards: not which model sounds best, but which automated judge best matches human judgement of expressive speech.
More leaderboards
Text-to-speech, scored on what it produces
Scored on acting, voice identity, and long-form stability.
Live voice agents, scored on emotional intelligence
Emotional intelligence in live, spoken conversation.
Speech understanding, beyond the words
Emotion, speaker, and synthetic detection - beyond the transcript.
Speech recognition, scored in the real world
Word error rate on accented, emotional, and talked-over speech.
Learn more
Introducing Real World VoiceEQ: Measuring the Human Quality of Voice AI
Jul 14, 2026
RW-Voice-EQ Bench: A Real World Benchmark for Evaluating Voice AI Systems
Current voice AI benchmarks typically evaluate isolated capabilities such as speech intelligibility, word error rate, or text-based dialogue quality, but they rarely test whether systems harness the acoustic information that distinguishes spoken language from its textual representation. To this end, we introduce the Real World Voice EQ Bench, a multidimensional benchmark for evaluating voice AI across text-to-speech (TTS), speech-to-speech (STS), speech understanding (SU), and automatic speech recognition (ASR). Our evaluations indicate that performance is highly dimension-specific. For TTS, naturalness, expressiveness, identity stability, and reliability are largely independent evaluation dimensions. For STS, access to audio does not guarantee use of vocal affect, and some agents remain largely transcript-driven. For SU, models perform unevenly across paralinguistic tasks. For ASR, real world accent, emotion, noise, and conversational conditions expose failures that are not captured by established clean-speech benchmarks. Together, these results show that voice AI should be evaluated as a profile of acoustic, expressive, interactional, and robustness capabilities rather than by a single aggregate score.
Put your judge on the board
Talk to our research team about measuring how well your automated judge tracks human ratings on expressive speech.