Many statistical models of interest in engineering, the sciences, and machine learning define a likelihood function that is computationally prohibitive to evaluate. This may be induced from the model only being known through a data generating process or the likelihood function involving a high-dimensional integral (e.g., from a marginalization procedure or the computation of a normalizing constant). In these cases, it is difficult to apply classical inference methods such as maximum likelihood estimation or likelihood-based Bayesian inference algorithms. To enable inference in these settings, several approaches have been developed in the statistics and machine learning community that avoid direct evaluation of the likelihood function (e.g., approximate Bayesian computation). Despite these success, efficiently solving such problems remains challenging, especially in high dimensions, or when only limited information or few samples are available. This mini-symposium will explore new algorithms and methodologies for performing likelihood-free inference in these complex models.
14:00
Robust Optimisation Monte Carlo
Michael Gutmann | University of Edinburgh | United Kingdom
Show details
Authors:
Michael Gutmann | University of Edinburgh | United Kingdom
Borislav Ikonomov | University of Edinburgh | United Kingdom
Approximate Bayesian Computation (ABC) is a framework to perform Bayesian inference when the likelihood function is intractable but simulating from the model is possible. While basic ABC algorithms are widely applicable, they are notoriously slow and much research has focused on increasing their efficiency. Optimisation Monte Carlo (OMC) has recently been proposed as an efficient and embarrassingly parallel method that leverages optimisation to accelerate the inference. We first demonstrate a previously unrecognised important failure mode of OMC: It generates strongly overconfident approximations by collapsing regions of similar or near-constant posterior density into a single point. We then propose an efficient, robust generalisation of OMC that corrects this. It makes fewer assumptions, retains the main benefits of OMC, and can be performed either as part of OMC or entirely as post-processing.
14:30
Sequential Neural Posterior Estimation for Likelihood-Free Inference
Jakob Macke | Technical University of Munich | Germany
Show details
Authors:
Jakob Macke | Technical University of Munich | Germany
David Greenberg | Technical University of Munich | Germany
Jan-Mathis Lueckmann | Technical University of Munich | Germany
Marcel Nonnenmacher | Technical University of Munich | Germany
Pedro Goncalves | caesar Bonn | Germany
Giacomo Bassetto | caesar Bonn | Germany
Theofanis Karaletsos | Uber AI Labs | United States
How can one perform Bayesian inference on stochastic simulators with intractable likelihoods? A recent approach is to learn the posterior from adaptively proposed simulations using neural network-based conditional density estimators, i.e. Sequential Neural Posterior Estimation (SNPE). I will review recent work on developing and improving SNPE approaches, and in particular `Automatic Posterior Transformation’ (APT). APT can modify the posterior estimate using arbitrary, dynamically updated proposals, and is compatible with powerful flow-based density estimators. It is more flexible, scalable and efficient than previous simulation-based inference techniques. I will show the power and versatility of this approach using dynamical models of biological systems.
15:00
Accelerating inference with measure transport and generative networks
Casey Dowdle | Cold Regions Research and Engineering Laboratory | United States
Show details
Authors:
Casey Dowdle | Cold Regions Research and Engineering Laboratory | United States
Matthew Parno | Cold Regions Research and Engineering Laboratory | United States
Conditional distributions are at the core of many uncertainty quantification techniques, including the exploration of Bayesian posteriors and characterizing Markov random fields. Sampling conditional distributions however, can be challenging when the conditional distribution is constructed from expensive nonlinear models or is only defined through historical data. Nonlinear random variable transformations, called transport maps, have recently emerged as a valuable tool in overcoming these challenges, but often rely on polynomial expansions and maximum likelihood loss functions that can have difficulty capturing complicated nonlinear correlations or multimodal structure. Paralleling the development of transport maps, there has been an explosion in machine learning techniques that employ data to construct generative models using neural networks. If constructed appropriately, these models can then be used for conditional sampling and tackle similar problems to transport maps. However, these techniques can be difficult to train and do not leverage more rigorous dimension reduction techniques found in the UQ community. We will explore the relationship between these parallel fields and introduce a framework that can leverage the flexibility of neural networks while also utilizing modern dimension reduction strategies. We will demonstrate the efficacy of this hybrid approach on several physics-based inverse problems.
15:30
Characterization and simulation of conditional probability densities
Giulio Trigila | Baruch College of New York | United States
Show details
Author:
Giulio Trigila | Baruch College of New York | United States
Conditional probability estimation provides data-based answers to critical questions, such as the expected response of specific patients to different medical treatments, weather forecasts and the effect of political measures on the economy. In the complex systems behind these examples, the outcome x of a process depends on many and diverse factors z. In addition, x is probabilistic in nature due in part to our ignorance of other relevant factors and to the chaotic nature of the underlying dynamics. This talk will describe a general procedure for the characterization, estimation and simulation of the conditional probability density ρ(x|z) underlying a sample set {xi,zi}. The methodology relies on a data-driven formulation of the Wasserstein barycenter problem, posed as a minimax problem in terms of two adversarial flows.