CS SEMINAR

Designing Unvoiced Interfaces for Silent, Secure, and Intuitive Interactions

Speaker
Shubham Jain, Assistant Professor, Department of Computer Science, Stony Brook University
Chaired by
Dr Ambuj VARSHNEY, Assistant Professor, School of Computing
ambuj@comp.nus.edu.sg

21 Jan 2025 Tuesday, 10:30 AM to 11:30 AM

MR21, COM3 02-61

Abstract:
In this talk, I will present a series of cutting-edge systems that leverage jaw motion to enable unvoiced interaction, a paradigm designed to overcome the limitations of voice-based technologies in noisy or privacy-sensitive environments. I will introduce our innovative ear-worn device that tracks jaw movement using twin IMUs to recognize unvoiced human commands. Our system breaks down jaw motion into constituent syllables and phonemes to recognize single word commands, outperforming traditional voice assistants in challenging acoustic environments. Next, I will present Unvoiced, which transforms jaw motion signals into audio spectrograms for silent interaction with devices supporting multiple applications. Finally, we explore the opportunities in using jaw motion as a biometric that can authenticate users based on jaw motion through inaudible speech recognition without microphones, resistant to both vocal imitations and replay attacks. Together, these systems pave the way for a new era of voiceless interactions, enhancing accessibility, privacy, and user convenience in diverse settings.

Bio:
Shubham Jain is an Assistant Professor in the Department of Computer Science at Stony Brook University where she leads PiCASSo (Pervasive Computing and Smart Sensing) Lab. Her research interests lie in cyber-physical systems, accessibility, and data analytics in smart environments. She is an NSF CAREER awardee and was also recognized as a rising star in networking and communication. She frequently publishes in top-tier venues. Her work on pedestrian safety has been featured in several media outlets, including The Wall Street Journal. She received her PhD in Electrical & Computer Engineering from Rutgers University in 2017.