Guaranteed Bounds for Posterior Inference in Universal Probabilistic Programming
Probabilistic Programming is a general-purpose means of expressing probabilistic models as programs, and automatically performing Bayesian inference. The vision of expressing probabilistic models as programs is elegant and unifying; and the construction of general-purpose Bayesian inference engines is attractive, as it can greatly improve access to machine learning algorithms. The reality is however somewhat different: existing inference algorithms have few guarantees on their results, and / or they only work on a restricted class of programs (models).
We propose guaranteed (stochastic and sound) bounds on the posterior distributions. Mathematically they are super/sub-additive measures which can be shown to sandwich the true posterior (of the program / model in question) with probability 1. They can be viewed as a (partial) correctness specification, and play a practically important role as diagnostics for the development of inference algorithms. Moreover they work for a very broad class of probabilistic programs, and constitute the basis for a new general-purpose inference algorithm.
We have built a tool implementation, called GuBPI, which automatically computes these posterior lower/upper bounds. Our evaluation on examples from the literature shows that the bounds are useful, and can be used to recognise wrong outputs from stochastic posterior inference procedures.
This talk will begin with an introduction to probabilistic programming.
Reference: Raven Beutner, Luke Ong, Fabian Zaiser: Guaranteed Bounds for Posterior Inference in Universal Probabilistic Programming. CoRR abs/2204.02948 (2022). https://doi.org/10.48550/arXiv.2204.02948
Prof Ong is a Distinguished University Professor and the Vice President of Research at NTU. He obtained his PhD in Computer Science from Imperial College, University of London. Prior to joining NTU, he was Lecturer then Professor of Computer Science at the University of Oxford and was also the Shaw Visiting Professor at NUS. Professor Ong's research ranges across semantics of computation, programming languages, verification, logic and algorithms, and algorithmic game theory.
Professor Ong has received several international and national accolades. He is the joint winner of the ACM / EATCS / EACSL / KGS Alonzo Church Award 2017 for Outstanding Contributions to Logic and Computation. He is also a recipient of the President of the Republic of Singapore Scholarship in 1981, Prime Minister's Book Prize in 1980, Overseas Merit Scholarship from 1981 to 1984.