In the last decades there has been renewed interest for Gaussian processes (GP) in statistics and machine learning. New challenges have arisen, especially in uncertainty quantification and optimization for complex systems. The case of continuous inputs has been intensively studied, and can be addressed with existing classes of GPs, such as isotropic (radial) kernels defined with the Euclidean distance. However, numerous applications involve more general non-Euclidean input spaces. This requires the definition of other GPs.
Fortunately, despite the diversity of situations, there are a few common techniques to define valid GPs, such as using a mapping to an Euclidean space. This mini-symposium aims at illustrating the variety of problems encountered along with their specific solutions, as well as the generic techniques. The first part, will focus on the case of discrete inputs in Gaussian process meta-modeling. By discrete input, we mean an input which has a finite number of levels, either ordered or not (it may also be called here “qualitative”, “categorical” or “factor” input). The second part, will present four other cases where the input space can be a permutation, time-varying, a probability distribution or a graph.
16:30
Gaussian processes on distributions
Jean-Michel Loubès | Institut de Mathématiques de Toulouse | France
Show details
Authors:
Jean-Michel Loubès | Institut de Mathématiques de Toulouse | France
François Bachoc | Institut de Mathématiques de Toulouse | France
Thi Thien Trang Bui | Institut de Mathématiques de Toulouse | France
Fabrice Gamboa | Institut de Mathématiques de Toulouse | France
Laurent Risser | Institut de Mathématiques de Toulouse | France
Patricia Balaresque | Laboratoire d’Anthropologie Moléculaire et Imagerie de Synthèse (AMIS), Faculté de Médecine Purpan | France
Nil Venet | Institut de Mathématiques de Toulouse & CEA | France
We consider computer code with distribution entries. To measure the variability of such input variables, we build a specific kernel using optimal transport methods. On the real line, we consider a kernel using Monge-Kantorovich (a.k.a Wasserstein) distance. For general distributions we use Wasserstein barycenters to design an efficient kernel which will be proved helpful to forecast new observations.
17:00
Multi-Fidelity modeling with varying input space dimensions using Deep Gaussian Processes
Ali Hebbal | ONERA & Université de Lille | France
Show details
Authors:
Ali Hebbal | ONERA & Université de Lille | France
Mathieu Balesdent | ONERA | France
Loic Brevault | ONERA | France
El-Ghazali Talbi | Université de Lille | France
Nouredine Melab | Université de Lille | France
Multi-fidelity approaches improve the inference of a high-fidelity (HF) model which is built using a small set of accurate observations, by taking advantage of its correlations with a low-fidelity (LF) model trained with a larger set of approximated data. Gaussian Processes (GPs) are usually used for multi-fidelity modeling [1]. Recently, Deep GPs [2] have been used to exhibit the correlations between LF and HF (MF-DGP) [3]. MF-DGP and the most existing multi-fidelity methods consider the inputs of LF and HF defined identically over the same input space. However, due to either different modeling approaches from one fidelity to another, or an omission of some variables in LF, the input spaces may differ in the form of the parametrization and also in dimensionality. In this talk, a new formulation of MF-DGP incorporating the mapping between the input spaces in a non-parametric way is proposed. The mapping between fidelities is within the multi-fidelity model, allowing a joint optimization of the mapping and of the multi-fidelity model. The case of vector-valued functions is also considered.
[1] M. C Kennedy and A. O'Hagan. Bayesian calibration of computer models. Journal of the Royal Statistical Society: Series B, 63(3):425-464, 2001
[2] A. Damianou and N. Lawrence. Deep gaussian processes. In Artificial Intelligence and Statistics, pp 207-215, 2013
[3] K. Cutajar, M. Pullin, et al. Deep gaussian processes for multi-fidelity modeling. ArXiv:1903.07320, 2019
17:30
Gaussian processes indexed on the symmetric group: prediction and learning
François Bachoc | Institut de Mathématiques de Toulouse | France
Show details
Authors:
François Bachoc | Institut de Mathématiques de Toulouse | France
Baptiste Broto | CEA | France
Fabrice Gamboa | Institut de Mathématiques de Toulouse | France
Jean-Michel Loubès | Institut de Mathématiques de Toulouse | France
In the framework of the supervised learning of a real function defined on a space X , the so called Kriging method stands on a real Gaussian field defined on X. The Euclidean case is well known and has been widely studied. In this paper, we explore the less classical case where X is the non commutative finite group of permutations. In this setting, we propose and study an harmonic analysis of the covariance operators that enables to consider Gaussian processes models and forecasting issues. Our theory is motivated by statistical ranking problems.
18:00
Variational inference for Gaussian Markov Random Fields
Nicolas Durrande | Prowler.io | United Kingdom
Show details
Authors:
Nicolas Durrande | Prowler.io | United Kingdom
Vincent Adam | Prowler.io | United Kingdom
Stefanos Eleftheriadis | Prowler.io | United Kingdom
James Hensman | Prowler.io | United Kingdom
Gaussian Markov Random Fields (GMRF) is a powerful framework for Gaussian process regression models defined on graphs. One of its key advantages is to benefit from the sparsity of the precision matrix of the Gaussian distribution to allow for fast evaluation of the model likelihood. We show in this work that Variational inference, which is typically used for approximating more complex models where the GP posterior distribution is intractable, can also benefit from the sparse precisions encountered in GMRF an lead to massive computational gains. We illustrate our approach on a GP classification and a Cox processes examples where the input space is a graph.