NewIntroducing the Hume Audio Decisions APIOne model, any question, straight from audio
hume.ai logo
Blogs
Research

Introducing the Hume ASR Entity Dataset

David Ayllon
Nathan Leung
David Ayllon and Nathan Leung
4 min read

Introducing the Hume ASR Entity Dataset

What do German singer Helene Fischer, British antiseptic brand Dettol, and the Mexican state of San Luis Potosí have in common? None of the 45 leading ASR systems we tested transcribed any of these three correctly.

ASR models continue to improve on standard benchmarks. But entities (the names, addresses, codes, dates, times, and amounts that tasks depend on) remain a persistent failure mode. In our recent analysis of entity performance in Real World VoiceEQ, even the best system missed about one in five entities.

In a voice agent, an entity error can derail the task. An incorrect email address or date of birth can block bank-account verification. A mistaken street address can misdirect a delivery. A mistranscribed passcode can prevent someone from turning off an alarm. For people with uncommon names or addresses, repeated failures can make voice AI effectively unusable.

One reason is the data. Common training sources such as audiobooks, podcasts, and news contain relatively few entities, relatively few non-native speakers, and mostly clean audio. ASR systems need targeted data to learn these harder cases.

Training for the words that matter most

The Hume ASR Entity Dataset is a custom dataset designed to post-train models on entities. It includes about 800 hours of scripted read speech in eight languages, with more than 350,000 tagged entities across 28 categories including:

  • person names, including athletes, as well as organizations, sports teams, brands, products, and food and drink names
  • street addresses, phone numbers, and email addresses
  • account numbers, identifiers, and alphanumeric codes
  • amounts, currencies, dates, times, and version numbers
  • spelled-out strings

The data spans the conversational domains where voice agents depend on exact transcription: support, sales, scheduling, travel, logistics, shopping, sports, food and drink, healthcare administration, media, education, government services, family, and personal use.

Built for real-world variation

We designed the dataset around failure modes surfaced by our own ASR evaluations, including foreign entities, accented speech, real acoustic conditions, and the gap between spoken and written forms.

Foreign entities in local speech

About 40% of the entities in each language come from another locale: a Mexican address read in English, a Japanese brand read in Spanish, a Turkish name read in German, or a British postcode read in Arabic. These combinations are common in real customer interactions and are where ASR systems fail most. Each entity is tagged with its locale of origin, so developers can split results between native and foreign entities.

Accent diversity

About 6,000 speakers from 49 countries took part in producing this dataset. Native speakers living abroad and speakers with regional accents are included alongside speakers in the home country. The English set alone includes speakers living in the United States, the United Kingdom, Canada, India, Egypt, Brazil, Mexico, Australia, and more.

Real acoustic conditions

Speakers recorded on their own laptops and phones, in their own homes and offices, using built-in microphones, headsets, and speakerphones. We applied no noise suppression or level normalization. Depending on the language, 5% to 17% of recordings have a signal-to-noise ratio below 15 dB, and another 21% to 30% fall between 15 and 25 dB, reflecting the conditions in which voice agents operate.

Labels for recognition and formatting

Each recording includes written, spoken, and tagged labels. The written label captures the formatted output a downstream system expects (“Book it for 04/18/2027 at 9:00 p.m.”); the spoken label records what was said (“book it for oh four eighteen two thousand and twenty-seven at nine p m”); and the tagged label marks each entity’s category and origin. Together, they let developers train recognition and formatting, separate pronunciation from text-normalization errors, and create speaker-disjoint train and test splits using anonymous speaker keys.

Quality control for accuracy

Because entity accuracy is the point of the dataset, every recording passed multiple quality checks. Recordings were made by people who were specifically recruited for the task. The clips were then checked for clipping, very low levels, silence, duplicates, missing speech, and other technical problems. To verify the recordings, each audio sample was transcribed independently and compared with the original script using multiple methods. Only recordings that passed every quality filter are included.

Hear the dataset

Recordings in four of the eight languages, each with a foreign-origin entity, shown with their written and spoken labels.

English: lgarcia@yahoo.com.br
0:00

English, a Brazilian email address.
Written: “Should I forward the contract to lgarcia@yahoo.com.br today?”
Spoken: “should I forward the contract to L Garcia at Yahoo dot com dot B R today”

Spanish: KE406922064GQ
0:00

Spanish, a Kenyan tracking number.
Written: “KE406922064GQ es el número de rastreo de mi compra.”
Spoken: “K E cuarenta sesenta y nueve veintidós cero seis cuatro G Q es el número de rastreo de mi compra”

German: VE06 GEX
0:00

German, a Polish license plate.
Written: “Der Fahrer mit VE06 GEX steht schon vor dem Restaurant.”
Spoken: “Der Fahrer mit V E null sechs G E X steht schon vor dem Restaurant”

Greek: 7:45
0:00

Greek, a New York time.
Written: “Ξεκινήστε την εκφόρτωση των κιβωτίων αύριο το πρωί ακριβώς στις 7:45 ώρα Νέας Υόρκης.”
Spoken: “ξεκινήστε την εκφόρτωση των κιβωτίων αύριο το πρωί ακριβώς στις οκτώ παρά τέταρτο ώρα Νέας Υόρκης”

Improve entity accuracy with purpose-built data

Built from the failure modes we see in leading ASR systems, the Hume ASR Entity Dataset gives developers targeted data for improving entity accuracy across languages, accents, and real acoustic conditions.

We offer the Hume ASR Entity Dataset for licensing to teams that want to train on these failure modes across languages, accents, and real recording conditions. We can also collect more hours for any language, add new languages, and build custom data for other domains. Talk to our team about licensing the dataset.

Appendix: Dataset contents

The Hume ASR Entity Dataset covers eight languages, with about 100 hours each. We can collect more hours for any of these languages and add new languages on request.

LanguageHoursRecordingsSpeakersEntitiesForeign origin
English~100~42,000~1,000~43,00048%
German~100~40,000~350~47,00042%
Spanish~100~37,000~700~43,00039%
Russian~100~40,000~200~43,00039%
Japanese~100~40,000~250~44,00038%
Arabic~100~40,000~900~44,00044%
Turkish~100~40,000~1,400~51,00042%
Greek~100~40,000~1,200~52,00037%

Delivery format

  • Audio: one 16 kHz, mono, 16-bit WAV per recording, converted to a standard format and trimmed of long leading and trailing silences without cutting quiet speech.
  • Manifest: JSONL and CSV files containing the written, spoken, and tagged labels; an anonymous speaker key; the domain; duration; and entity count.
  • README: a dataset card covering speaker countries, age range, gender balance, and noise profile.

Related topic

View more

Build voice models that understand more than words.