Johns Hopkins University researchers have created a system that could make social robots more effective at detecting and managing user interruptions in real time based on a human speaker's intent—a breakthrough for areas like health care and education where natural conversation is crucial. The team presented its work at this year's Robotics: Science and Systems conference, held in Los Angeles June 21 to 25.
Despite all their advancements, state-of-the-art robotic systems still have difficulties handling user interruptions in real time—and often don't understand why humans interrupt in the first place.
So Jiwon Moon, Engr '25; computer science PhD students Shiye "Sally" Cao, Amama Mahmood, and Victor Nikhil Antony; and assistant professors Ziang Xiao, Anqi "Angie" Liu, and Chien-Ming Huang began by analyzing different types of human conversations like discussions, talk show interviews, and press briefings to identify how humans handle someone talking over them.
The researchers observed that interruptions can have different purposes, such as signaling understanding, assisting the speaker, seeking clarification, expressing disagreement, further developing the topic, or changing the subject. Likewise, those who were interrupted reacted in various ways: ignoring the interruption, acknowledging it but continuing, or yielding to the interrupter.
Using these patterns, the team developed a robotic interruption handling system that uses large language models (LLMs) to adopt different conversational strategies based on the predicted intention of the interrupter.
"To the best of our knowledge, this is the first robotic system that has integrated intention classification into its real-time interruption-handling framework," says first author Cao. "By categorizing human interruptions into four categories—agreement, assistance, clarification, and disruption—our system tailors handling strategies to match the context and user intention behind the interruption."
The system works by first detecting overlapping speech in a conversation and sending the content of that interruption to an LLM. The LLM then determines the intent behind the interruption and decides on one of several handling strategies.
When the human interrupter is agreeing with or assisting the conversation, the robot acknowledges this, nods, and resumes speaking. If the interrupter is asking for clarification, the robot supplies it before continuing. And for more disruptive interruptions, such as those that derail the conversation or change the subject, the robot has two options: It can either hold the floor to summarize its remaining points before yielding to the human user, or it can stop talking immediately.
The team integrated this system into a social robot and conducted a user study to assess its ability to identify different kinds of interruptions. The system accurately classified the underlying intention behind 88.78% of interruptions and successfully handled them 93.69% of the time, the researchers report.
"Interestingly, while holding the floor is a common strategy for humans to handle interruptions, our study participants didn't always like it when the robot did so," Cao says. "They perceived its role as assistive, rather than collaborative, and so they expected it to yield immediately to them at every turn. This shows the importance of aligning a robot's role and task context with its interruption handling behavior."
The researchers suggest this may help reinforce a robot's intended role in a given context. For example, a robot designed to be an assistive tool in an informal task should adopt a more flexible and accommodating approach to handling interruptions.
The team also recommends that future work explore non-verbal interruptions—like a user opening their mouth to speak without saying anything—and investigate interruption handling in longer or multi-session interactions with multiple users.
"When effectively used and handled, interruptions can lead to fluid and fast-paced conversations—but if they are managed inadequately, they can disrupt conversational flow, cause breakdowns, and make interrupters feel excluded," Cao says. "If conversational robots are to become assistants, teammates, and companions in people's everyday lives, it is critical that they are able to detect and manage interruptions on-the-fly. Our work is an initial step towards that capability."
This research was supported by the National Science Foundation.
Posted in Science+Technology
