Reproducing kernel Hilbert spaces are ubiquitous in applied mathematics and statistics due to the tractability provided by the reproducing property. They commonly underpin the theoretical analysis of stochastic processes (including Gaussian processes) and have historically been used to construct a variety of numerical schemes for interpolation, integration or solving differential equations. Additionally, within the machine learning literature, the last decade has seen fruitful research into the use of kernels in statistical tests, statistical estimators and sampling methods.
In this two-part mini-symposium, we propose to explore these more recent works and highlight their relevance to uncertainty quantification. The first session will focus on the use of kernel-based probability metrics and statistical divergences to construct statistical estimators and hypothesis tests for high-dimensional models or models with intractable likelihoods. The second session will focus on applications of kernels to problems in Monte Carlo methods and approximation of probability measures.
14:00
Learning Laws of Stochastic Processes
Harald Oberhauser | University of Oxford | United Kingdom
Show details
Author:
Harald Oberhauser | University of Oxford | United Kingdom
The signature map provides a natural notion of "polynomials on path space" and led to much progress in stochastic analysis. More recently, it has found applications in machine learning for sequence-valued data, such as time-series. I will discuss how this approach can be combined with classic ideas from kernel learning to define a MMD metric for laws of stochastic processes that has several desirable properties. En passant, this requires to answer some questions about (kernel-)learning on non-compact spaces. Further, I will discuss applications in a Bayesian/GP setting and how to deal with the computational complexity.
14:30
Statistical Inference for Generative Models with Maximum Mean Discrepancy
Alessandro Barp | University of Cambridge & Imperial College London | United Kingdom
Show details
Author:
Alessandro Barp | University of Cambridge & Imperial College London | United Kingdom
While likelihood-based inference and its variants provide a statistically efficient and widely applicable approach to parametric inference, their application to models involving intractable likelihoods poses challenges. In this work, we study a class of minimum distance estimators for intractable generative models, that is, statistical models for which the likelihood is intractable, but simulation is cheap. The distance considered, maximum mean discrepancy (MMD), is defined through the embedding of probability measures into a reproducing kernel Hilbert space. We study the theoretical properties of these estimators, showing that they are consistent, asymptotically normal and robust to model misspecification. A main advantage of these estimators is the flexibility offered by the choice of kernel, which can be used to trade-off statistical efficiency and robustness. On the algorithmic side, we study the geometry induced by MMD on the parameter space and use this to introduce a novel natural gradient descent-like algorithm for efficient implementation of these estimators. We illustrate the relevance of our theoretical results on several classes of models including a discrete-time latent Markov process and two multivariate stochastic differential equation models.
15:00
Kernel-based Statistical Tests on Infinite Dimensional Data
George Wynne | Imperial College London | United Kingdom
Show details
Author:
George Wynne | Imperial College London | United Kingdom
This talk considers generalisations of kernel-based statistical tests. These tests have proven extremely powerful in applications to low-dimensional data, but suffer from a curse of dimensionality which limits their applicability to high- or infinite-dimensional problems such as time series or stochastic processes. In this talk, we propose to generalise classical kernel-based statistical tests to define them directly in function space and hence limit the impact of dimensionality on the power of the test. Our approach will focus around a natural generalisation of the Gaussian kernel to infinite dimensional Hilbert spaces, which shall facilitate closed form expressions for a wide class of examples.
15:30
- NEW - Conditional quantile function estimation as an infinite task learning problem
Alex Lambert | Télécom Paris | France
Show details
Authors:
Alex Lambert | Télécom Paris | France
Romain Brault | Thales | France
Zoltan Szabo | École Polytechique | France
Maxime Sangnier | Sorbonne University | France
Florence D'alche-Buc | Telecom ParisTech | France
Conditional quantile functions provide a widely-used tool to quantify uncertainty in statistics. While classical approaches tackle the task using a fixed quantile level, in this work we present an approach capable of jointly learning a continuum of conditional quantile functions. In order to achieve this goal, we leverage operator-valued kernels and their associated vector-valued RKHSs to model functional outputs and rely on the minimization of an integrated pinball loss. We provide generalization guarantees to the suggested scheme and illustrate its numerical efficiency in several quantile benchmarks.