Talk 1: Benefits of Structure and Randomness in Deep Learning
Talk 2: Accelerating Science through Knowledge-Guided AI
2) Vipin Kumar, Regents Professor and William Norris Land Grant Chair in Large-Scale Computing, University of Minnesota
COM3 Level 1
SR13, COM3 01-22
Talk 1:
Abstract:
We consider the high-dimensional geometry associated with the first and second order structure, i.e., gradients and Hessians, of deep learning (DL) losses. We start with a review of ongoing work which is starting to highlight the unique geometry associated with spectral decay of such gradients and Hessians.
In the talk, we discuss three lines of advances which are utilizing such structure along with randomness. First, we revisit the challenges of differentially private (DP) DL, and demonstrate how DP-DL with random gradient projections using the first order geometry hold considerable promise. Second, we revisit the challenges of distributed and federated DL, especially the communication costs of high-dimensional gradients. We show that random gradient projection can be effective for communication efficient DL with convergence determined by the intrinsic dimension of the loss Hessians, which is substantially smaller than the high ambient dimension. Third, we consider sequential decision making with neural policies and, surprisingly, establish O(log T) regret bounds even though the problem does not fall under the standard online convex optimization paradigm. The development utilizes the Hessian geometry of DL, along with some unique use of random perturbations.
Bio:
Arindam Banerjee is a Founder Professor at the School of Computing and Data Science, University of Illinois Urbana-Champaign. He currently serves as the President of the Society for Artificial Intelligence and Statistics which runs the annual AISTATS conferences and is an ACM Fellow. His research interests are in machine learning and artificial intelligence. His current research focuses on computational and statistical aspects of over-parameterized models including deep learning, spatial and temporal data analysis, generative models, and sequential decision making problems. His work also focuses on applications of machine learning in complex real-world and scientific domains including problems in climate science, ecology, and finance. He has won several awards, including the NSF CAREER award (2010), the IBM Faculty Award (2013), and seven best paper awards at top-tier venues.
Talk 2
Abstract:
Machine learning (ML), a key driver of recent advances in AI, has achieved remarkable success in fields such as computer vision and language modeling, and the scientific community is increasingly eager to harness its potential for addressing critical societal challenges. However, realizing this promise requires a paradigm shift in data-intensive scientific discovery, as “black box” ML models often fail to generalize to unseen scenarios and can yield results that conflict with established scientific understanding.
This talk will focus on knowledge-guided machine learning (KGML), a leading approach within the broader vision of knowledge-guided AI. KGML embeds scientific principles directly into model design and training, producing models that are more robust, generalizable, and consistent with the laws of nature. These methods are particularly relevant for scientific and engineering problems traditionally addressed by process-guided (mechanistic or first principle–based) models, where progress is often limited by incomplete or uncertain knowledge of underlying processes.
Illustrative examples will be drawn from environmental sciences (e.g., ecology, hydrology, and agronomy), but the broader significance of KGML lies in its generality: it offers a powerful paradigm for advancing discovery across any discipline where mechanistic models play a central role.
Bio:
Vipin Kumar is a Regents Professor and William Norris Endowed Chair in the Department of Computer Science and Engineering at the University of Minnesota. He is internationally recognized for pioneering contributions in artificial intelligence, machine learning, and high-performance computing, and their applications to critical societal challenges such as climate change, ecosystems, and health care.
Kumar’s early research on parallel AI search led to the development of the isoefficiency metric, a foundational framework for analyzing the scalability of parallel algorithms. The concept has influenced the design of scalable algorithms across a wide range of applications including molecular dynamics, structural mechanics, fluid dynamics, and training of large-scale neural networks. His group’s work on graph partitioning led to widely used software such as METIS, ParMETIS, and hMETIS, which remain staples in domains ranging from circuit design and scientific simulations to large-scale graph analytics. These advances also opened the door to scalable approaches in data mining, where his group produced highly cited contributions in clustering, association analysis, and anomaly detection.
Over the past two decades, Kumar’s research has focused on advancing AI and data science to address global environmental challenges, including climate change, biodiversity loss, and food and water security. His work on massive Earth observation datasets has produced global-scale insights on land cover change (e.g., deforestation), forest fire activity, and surface water dynamics. This information provides critical input for understanding human and climate impacts on the environment and informing international climate assessments. About a decade ago, this trajectory led to the creation of knowledge-guided machine learning (KGML), a rapidly growing field with applications in aquatic sciences, hydrology, and agriculture.
He has coauthored more than 400 papers and 11 books, including Introduction to Parallel Computing, Introduction to Data Mining, and Knowledge-Guided Machine Learning. He is a Fellow of AAAI, AAAS, ACM, IEEE, and SIAM, and his honors include the ACM SIGKDD Innovation Award (2012), the IEEE Sidney Fernbach Award (2016), the SC Test-of-Time Award (2021), and the IEEE Taylor L. Booth Education Award (2025).