The task of processing large amounts of data in order to model complex associated dynamical systems is an important challenge of the 21st century. The need for novel mathematical concepts and advanced computational techniques in this context has accelerated research in the associated fields of Data Assimilations and Machine Learning. In recent years the two research communities have been growing closer resulting in advanced numerical methods that combine the strength of both worlds and the development of theoretical underpinning of existing and new techniques. The aim of this MS is to foster these emerging bridges, to detect limitations and possible future alleys by bringing together people from both communities and creating a room for scientific exchange.
10:30
Kalman-Wasserstein Gradient Flows
Franca Hoffmann | California Institute of Technology | United States
Show details
Author:
Franca Hoffmann | California Institute of Technology | United States
We study a class of interacting particle systems that may be used for optimization. By considering the mean-field limit one obtains a nonlinear Fokker-Planck equation. This equation exhibits a novel gradient structure in probability space, based on a modified Wasserstein distance which reflects particle correlations: the Kalman-Wasserstein metric. This setting gives rise to a methodology for calibrating and quantifying uncertainty for parameters appearing in complex computer models which are expensive to run, and cannot readily be differentiated. This is achieved by connecting the interacting particle system to ensemble Kalman methods for inverse problems. This is joint work with Alfredo Garbuno-Inigo (Caltech), Wuchen Li (UCLA) and Andrew Stuart (Caltech).
11:00
Quantifying ensemble Kalman inversion as a derivative-free optimizer
Neil Chada | NUS | Singapore
Show details
Author:
Neil Chada | NUS | Singapore
Many data science problems can be formulated as an inverse problem, where the parameters are estimated by minimizing a proper loss function. When complicated black-box models are involved, derivative-free optimization tools are often needed.The ensemble Kalman filter (EnKF) is a particle-based derivative-free Bayesian algorithm originally designed for data assimilation. Recently, it has been applied to inverse problems for computational efficiency. The resulting algorithm, known as ensemble Kalman inversion (EKI), involves running an ensemble of particles with EnKF update rules so they can converge to a minimizer. In this talk, we investigate EKI convergence in general nonlinear settings. To improve convergence speed and stability, we consider applying EKI with non-constant step-sizes and covariance inflation. We prove that EKI can hit critical points with finite steps in non-convex setting. We also prove that EKI converges to the global minimizer polynomially fast if the loss function is strongly convex. We verify the analyses presented with numerical experiments on various inverse problems.
11:30
Nonlinear ensemble filtering and smoothing via couplings
Ricardo Baptista | Massachusetss Insitute of Technology | United States
Show details
Author:
Ricardo Baptista | Massachusetss Insitute of Technology | United States
We consider the Bayesian filtering and smoothing problems for high-dimensional non-Gaussian state-space models with challenging nonlinear dynamics and sparse measurements in space and time. While the ensemble Kalman filter and smoother yield robust approximations to the distributions of interest, these algorithms are limited by linear transformations and are generally inconsistent with the Bayesian solution in the large-sample limit. To generalize these methods, we propose a methodology that transforms the non-Gaussian ensemble at each assimilation step into samples from the current filtering or smoothing distribution via a sequence of nonlinear couplings. These couplings are based on transport maps that can be computed quickly using convex optimization and avoid any form of importance sampling, making the approach also applicable for inference in likelihood-free settings. In this presentation, we explore the low-dimensional structure in the couplings that is inherited from the sequential inference problem (e.g., decay of correlation, conditional independence, and local likelihoods). We exploit this structure to regularize the map estimation in high dimensions and reduce its ensemble size requirements. The numerical performance of our algorithms will be presented in the context of chaotic dynamical systems (e.g., the Lorenz 96 model).
12:00
Graph methods for semi-supervised learning and Bayesian inverse problems
Daniel Sanz-Alonso | University of Chicago | United States
Show details
Author:
Daniel Sanz-Alonso | University of Chicago | United States
In this talk I will consider two graph-based learning problems. The first one concerns a graph formulation of Bayesian semi-supervised learning, and the second one concerns kernel discretization of Bayesian inverse problems on manifolds. I will show that understanding the continuum limit of these graph-based problems is helpful in designing sampling algorithms whose rate of convergence does not deteriorate in the limit of large number of graph nodes.