A team of researchers from Cornell Tech is reimagining how technology can assist people with speech disabilities. Their focus is not only on improving functional speech but also on enabling users to make real-time jokes during conversations.
This research explores how artificial intelligence (AI) can enhance the experience for users of augmentative and alternative communication (AAC) devices, allowing them to deliver humorous remarks. Traditional AAC systems are often slow, making it difficult for users to match the timing of conversations, which can limit their ability to engage in humor.
“Humor is a vital part of social interaction, but for someone with a speech disability, timing it right is a challenge,” said Tobias Weinberg, the lead author of the paper. Weinberg, a Ph.D. student at Cornell Tech, lost his ability to speak at 15 and now uses AAC technology. “When I lost my ability to speak, I had to adapt my sense of humor to a new way of communicating. This research reflects that journey.”
The paper, titled “Why So Serious? Exploring Timely Humorous Comments in AAC Through AI-Powered Interfaces,” won two awards at the Association for Computing Machinery’s CHI ’25 conference, held in Yokohama, Japan, from April 26 to May 1.
The study involved collaboration between Cornell Tech, the Cornell Ann S. Bowers College of Computing and Information Science, and assistant professor Stephanie Valencia from the University of Maryland.
Weinberg explained, “There’s a trade-off between efficiency and control. A button that automatically generates jokes is fast, but they may not be your own. On the other hand, typing out the whole message gives you control but is slow, and you might miss the timing. We explored how to balance these factors to guide AI in creating the jokes users want.”
The study introduced participants with speech disabilities to four AI-assisted interfaces: “Full-auto,” “Keywords,” “Context Bubble Selection,” and “Wizard.” Each option offered a different balance of speed and control, letting users either quickly generate jokes or fine-tune them to suit their style.
The “Full-auto” interface offers minimal user input, allowing the AI to generate jokes automatically. The “Keywords” option lets users choose guiding terms to influence the AI’s content. The “Context Bubble” interface enables users to highlight conversation points for the AI to focus on, followed by keyword refinement. The “Wizard” interface provides the most customization, breaking down conversations into keywords and letting users select related ideas.
“The trade-off between control and timing affects all AI-based tools,” said Thijs Roumen, project advisor and assistant professor at Cornell Tech. “We saw this trade-off play out differently depending on the user group.”
Before testing the tools, participants preferred the option to craft jokes themselves. However, after using the tools, they favored more time-efficient interfaces.
“The window for delivering a humorous comment is very short,” Weinberg said. “Even if the joke isn’t fully my own, it still serves its purpose—making people laugh.”
The study also highlighted challenges, such as difficulties for users relying on eye-gaze or joystick inputs to scroll, as well as the lack of emotional depth in text-to-speech voices.
Weinberg emphasized that there’s no one-size-fits-all solution for improving AAC devices. “Each user has different preferences, which may even change depending on who they’re talking to.”
Roumen added, “Most research focuses on the direct needs of people with disabilities, but self-expression, humor, and emotional connection are often overlooked. I hope this study encourages more research in this area.”
The paper calls for a broader focus on accessibility in research. “We need to enhance expressiveness in AAC communication, allowing people with speech impairments to fully participate in conversations and thrive in society,” Weinberg concluded.
