Transport maps are deterministic couplings between probability measures with broad applications in uncertainty quantification and machine learning. They have been used for posterior sampling in Bayesian inference, for accelerating Markov chain Monte Carlo and importance sampling algorithms, and as building blocks of generative models and density estimation methods. More broadly, transport---including but not limited to optimal transport---provides an important mathematical foundation for many tools in machine learning and uncertainty quantification. The recent surge of interest in transport maps has been accompanied by efficient numerical methods that make constructing and learning such maps tractable in high dimensions and for large data sets. This minisymposium brings together researchers from uncertainty quantification and machine learning to discuss recent advances in theory, numerics, and applications of transport maps and related techniques.
14:00
Sum-of-squares polynomial flow
Yaoliang Yu | University of Waterloo | Canada
Show details
Author:
Yaoliang Yu | University of Waterloo | Canada
Triangular map is a recent construct in probability theory that allows one to transform any source probability density function to any target density function. Based on triangular maps, we propose a general framework for high-dimensional density estimation, by specifying one-dimensional transformations (equivalently conditional densities) and appropriate conditioner networks. This framework (a) reveals the commonalities and differences of existing autoregressive and flow based methods, (b) allows a unified understanding of the limitations and representation power of these recent approaches and, (c) motivates us to uncover a new Sum-of-Squares (SOS) flow that is interpretable, universal, and easy to train. We perform several synthetic experiments on various density geometries to demonstrate the benefits (and shortcomings) of such transformations. SOS flows achieve competitive results in simulations and several real-world datasets.
14:30
Projected Stein variational Newton: A fast and scalable Bayesian inference method in high dimensions
Peng Chen | Oden Institute, UT Austin | United States
Show details
Author:
Peng Chen | Oden Institute, UT Austin | United States
In this talk, I present a projected Stein variational Newton (pSVN) method for high-dimensional Bayesian inference. To address the curse of dimensionality, we exploit the intrinsic low-dimensional geometric structure of the posterior distribution in the high-dimensional parameter space via its Hessian (of the log posterior) operator and perform a parallel update of the parameter samples projected into a low-dimensional subspace by an SVN method. The subspace is adaptively constructed using the eigenvectors of the averaged Hessian at the current samples. We demonstrate fast convergence of the proposed method, complexity independent of the parameter and sample dimensions, and parallel scalability.
15:00
- CANCELED - Minimum Stein Discrepancy Estimators and their Gradient Flows
Francois-Xavier Briol | University College London | United Kingdom
Show details
Author:
Francois-Xavier Briol | University College London | United Kingdom
A common issue for maximum likelihood estimation in UQ is the lack of tractability of certain likelihood functions, with one major issue being unknown normalisation constants due to the integration of latent variables. In those cases, statisticians and machine learners often turn to methods such as score matching, contrastive divergence, or
minimum probability flow to obtain tractable parameter estimates. We provide a unifying perspective of these techniques as minimisers of a Stein discrepancy. We then propose a general class of "minimum Stein discrepancy estimators" from which we can design estimators with complementary strengths. On the theoretical side, we establish the consistency, asymptotic normality, and robustness of these estimators. On the practical side, we propose a stochastic Riemannian gradient descent algorithm which approximates the corresponding gradient flow and provides a tractable implementation of the methodology. Overall, the flexibility of this new class of estimators can help us tackle new problems including inference for non-smooth or heavy-tailed densities, and inference in the presence of outliers or corrupted data.
15:30
Generative models for Bayesian inference
Uros Seljak | University of California at Berkeley | United States
Show details
Author:
Uros Seljak | University of California at Berkeley | United States
The goal of generative models is to learn high dimensional analytic probability distribution from the samples, which can be data, simulations, or MCMC posterior samples of parameters. In high dimensions and for a large number of samples this can be a very difficult task, and I will describe a new approach to this problem that differs from the current methods popular in machine learning. I will then present applications of the method, of which the most novel is the use of these methods for Bayesian evidence. Bayesian evidence and Bayes factor is commonly used for Bayesian hypothesis testing, but current approaches based on nested sampling or annealed importance sampling are extremely expensive and often inaccurate. I will present tests on difficult high dimensional examples such as thin bananas or multi-modal posteriors, which show that our method achieves significantly higher accuracy than existing methods, at a small fraction of their computational cost. These methods are particularly powerful in situations where the likelihood evaluation is very expensive, a common situation in scientific data analysis. I will also present applications of the method to generative learning and generative sampling.