How Dot uses Hume's API for emotionally intelligent AI

Nuances of speaking to an AI with voice

Dot, a personal AI developed by New Computer, is designed to grow alongside its users, serving as a thought partner, friend, and confidant for young adults navigating uncertainty in their lives. Built on an infinite long-term memory system, Dot's responses become increasingly personalized with each interaction, whether through speaking or typing.

After introducing a voice message feature, New Computer noticed that Dot interpreted these messages too literally, relying solely on transcriptions. According to Luca Beetz, Founding AI Engineer, this led to “flat, robotic conversations where Dot often misunderstood the user’s intent.” For example, Dot would respond with the same standard helpfulness to a question posed in a defeated tone as it would to one posed in an inquisitive tone.

To truly understand users, Dot needed to be able to pick up on nonverbal cues, so the New Computer team turned to Hume for a scientifically-grounded technical solution.

Finding a solution in Hume's emotionally intelligent AI

To recognize users’ moods, New Computer integrated Hume’s Expression Measurement API. Hume’s expression measurement models – which are built on a decade of research, millions of proprietary data points, and over 30 publications in leading journals – allow Dot to interpret and respond with emotional intelligence to 25 distinct emotions by analyzing vocal tones. 

New Computer found integrating the API “extremely easy,” taking “less than a day to integrate and get up and running.”

The impact of empathic AI

• Hume’s Expression Measurement APl allows Dot to maintain the emotional context of conversations in its long-term memory, enabling it to build a ‘theory of mind’ about its users and how they respond to different topics or under various circumstances. 

• Dot can respond to messages with greater emotional intelligence, making interactions feel more human and empathetic. This builds trust and paves the way for a real relationship to be built, one that moves beyond the transactional messaging interaction that exists between most users and their AI tools today.

• By detecting and responding to emotional expressions, Dot boosts user engagement and satisfaction, making the app a trusted companion.

Since integrating Hume's Expression Measurement API, over 60% of users have engaged with Dot using voice messages. 

“We’ve seen that users want to be able to interact with Dot over voice, like they would with a friend,” says Samantha Whitmore, CEO of New Computer. “Thanks to the emotional interpretations we get from Hume, users are consistently amazed by Dot’s ability to intuitively sense and respond to the emotional undertones in their messages.”

Creating a sense of true, empathetic understanding pays off: over 45% of users that chat with Dot consistently convert to paid subscriptions. New Computer attributes much of this high conversion rate to the sense of being seen and heard that Dot provides through its emotionally intelligent interactions.

Learn More: Start measuring vocal and facial expressions with unmatched precision using Hume’s Expression Measurement API. Instantly capture nuanced expressions in audio, video, and images, such as awkward laughter, sighs of relief, and nostalgic glances. Enhance your product decisions and user experience with models based on 10+ years of research. Start building today.