Recent Advances in Emotion- and Intelligence-Integrated Audio-Visual Content Generation
AS6 Level 5
MR6, AS6-05-10
Abstract:
With the rapid development of Artificial Intelligence Generated Content (AIGC), audio-visual content generation has evolved toward cross-modal and emotion-aware intelligence, driven by increasing demands in film production, interactive media, and digital education. This talk presents recent progress in audio-visual content generation, including humanized audiobook generation, long-form video dubbing, and music generation for dance videos. In addition, I introduce advances in Chinese Cued Speech generation and recognition, highlighting a cross-modal mutual learning framework based on parameter-efficient low-rank Transformers for efficient multimodal interaction, as well as the integration of chain-of-thought prompting and diffusion models to improve gesture accuracy and robustness.
Bio:
Li Liu is currently an Assistant Professor at The Hong Kong University of Science and Technology (Guangzhou). She earned her Ph.D. from Université Grenoble Alpes and CNRS, France. Her primary research interests include audio-visual speech recognition and generation, multimodal learning, and artificial intelligence. She has published over 70 papers as first or corresponding author in top-tier journals and conferences in the field, including TPAMI, TMM, TASLP, NeurIPS, AAAI, ACM MM, and ICASSP. She currently serves as the Chair of the Member Nominations & Election Subcommittee of the IEEE Machine Learning for Signal Processing (MLSP) Technical Committee. She has also held roles as Local Chair (China site) for ICASSP 2022. As PI, she leads multiple research projects, including a sub-project of the NSFC Key Program, NSFC General and Young Scientist Programs, the 2024 CCF-Tencent Rhino-Bird Research Program, the 2023 and 2025 Tencent AI Lab Rhino-Bird Special Programs, the 2025 CCF-Kuaishou Foundation Model Explorer Fund, etc. She has been recognized with the Guangdong Provincial Outstanding Young Talent, Guangzhou Outstanding Young Talent, French Sephora Berribi Women Scientist Award, the IEEE Multimedia Signal Processing Rising Star Runner-up Award, and the 2024 CCF-Tencent Rhino-Bird Program Excellence Award.