Inside a crowded bar, even the best noise-canceling earbuds struggle. They can either shut the whole world out or let everything in, but they can’t do what humans do naturally: focus on the voices that matter while ignoring everything else. A new study from researchers at the University of Washington proposes a third way—a “proactive hearing assistant” that automatically figures out who you’re talking to using AI and enhances only their voices in real time, without taps or gestures.
“We were asking a very simple question,” says Shyam Gollakota, head of the Mobile Intelligence Lab at the University of Washington and coauthor of the study. “If you’re in a bar with a hundred people, how does the AI know who you are talking to?”
The team’s answer blends audio engineering with conversational science. Building on previous research by Gollakota’s lab, the system uses AI trained to detect the subtle turn-taking patterns humans instinctively follow to alternate speaking turns with minimal overlap. That conversational rhythm becomes the cue for identifying who is in the exchange. Voices that don’t follow the pattern are filtered out.
The prototype uses microphones in both ears and a directional audio filter aimed at the wearer’s mouth to extract the user’s own speech, which acts as an anchor for detecting turn-taking. With that anchor, the system isolates and enhances conversation partners while suppressing everyone else, operating at latencies less than ten milliseconds—fast enough to keep the amplified audio aligned with lip movements.
“The key insight is intuitive,” Gollakota says. “If I’m having a conversation with you, we aren’t talking over each other as much as people who are not part of the conversation.” The AI identifies voices that alternate naturally with the wearer’s own and ignores those that overlap too often to fit the conversation. The method does not rely on proximity, loudness, direction, or pitch. “We don’t use any sensors beyond audio,” he says. “You could be looking away, or someone farther away could be speaking louder—it still works.”
The technology could be useful to people who have hearing challenges, as traditional hearing aids amplify all sound and noise alike. “It could be extremely powerful for quality of life,” says Gollakota. Proactive hearing assistants with this technology could also help older users who would struggle to manually select speakers to amplify.
To deal with latency issues, the system uses a two-part model that mimics how our brains also process conversation. Shyam Gollakota
A Brain-Inspired Dual Model
To feel natural, conversational audio must be processed in under ten milliseconds, but detecting turn-taking patterns requires one to two seconds of context. Reconciling those timescales required a split architecture: a slower model that updates once per second and a faster model that runs every 10 to 12…
Read full article: AI Enhances Hearing Devices for Clearer Conversations
The post “AI Enhances Hearing Devices for Clearer Conversations” by Meghie Rodrigues was published on 12/08/2025 by spectrum.ieee.org