Safe and Efficient Exploration in Model-based Deep Reinforcement Learning
COM3 Level 1
SR12, COM3 01-21
Abstract:
How can we enable systems to efficiently and safely learn online, from interaction with the real world? I will first discuss safe Bayesian optimization, where we quantify uncertainty in the unknown reward function and constraints, and, under some regularity conditions, can guarantee both safety and convergence to a natural notion of reachable optimum. I will then consider Bayesian model-based deep reinforcement learning, where we use the epistemic uncertainty in the dynamics model to guide exploration while ensuring safety. Lastly I will discuss how we can leverage data-driven priors pre-trained from related tasks and simulations, and discuss several applications in robotics, science and language models.
Biography:
Andreas Krause is a Professor of Computer Science at ETH Zurich, where he leads the Learning & Adaptive Systems Group. He also serves as Academic Co-Director of the Swiss Data Science Center and Chair of the ETH AI Center, and co-founded the ETH spin-off LatticeFlow. Before that he was an Assistant Professor of Computer Science at Caltech. He received his Ph.D. in Computer Science from Carnegie Mellon University (2008) and his Diplom in Computer Science and Mathematics from the Technical University of Munich, Germany (2004). He is a Max Planck Fellow at the Max Planck Institute for Intelligent Systems, ACM Fellow, IEEE Fellow, ELLIS Fellow, a Microsoft Research Faculty Fellow and a Kavli Frontiers Fellow of the US National Academy of Sciences. He received the Rössler Prize, ERC Starting Investigator and ERC Consolidator grants, the German Pattern Recognition Award, an NSF CAREER award as well as the ETH Golden Owl teaching award. His research on machine learning and adaptive systems has received awards at several premier conferences and journals, including the ACM SIGKDD Test of Time award 2019 and the ICML Test of Time award 2020.