Model Merging with Sparsity: Theory, Algorithms and Applications
Abstract:
In this talk, I will be discussing model merging, a new paradigm of post-training algorithms for multi-task learning with large-scale models. I will begin by introducing its application in safe information localization of LLaMa 3 models, and then discuss the underlying ideas that enable the design of efficient and sparse localization algorithms. I will conclude the talk by discussing a theory to explain its practical success, which also shed light on the fundamental factors that enable its wide applications.
Bio:
Dr. Han Zhao is an Assistant Professor of Computer Science and, by courtesy, of Electric and Computer Engineering at the University of Illinois Urbana-Champaign (UIUC). He is also an Amazon Visiting Academic at Amazon AI. Dr. Zhao earned his Ph.D. degree in machine learning from Carnegie Mellon University. His research interest is centered around trustworthy machine learning, with a focus on algorithmic fairness, robust generalization and model interpretability. He has been named a Kavli Fellow of the National Academy of Sciences and has been selected for the AAAI New Faculty Highlights program. His research has been recognized through a Google Research Scholar Award, an Amazon Research Award, and a Meta Research Award.