Historically, design and analysis of computer experiments focused on deterministic solvers from the physical sciences via Gaussian process (GP) interpolation. But nowadays computer modeling is common in the social, management and biological sciences, where stochastic simulations abound. In this minisymposium, we bring together a selection of researchers in the areas of statistical surrogate modeling, active learning, and Bayesian optimization of stochastic computer model, simulation campaigns, and high volume observational studies. Noisier simulations demand bigger experiments to isolate signal from noise, and more sophisticated GP models -- such as adding a variance processes to track changes in noise throughout the input space in the face of heteroskedasticity. Appropriate surrogate modeling is key to the propagation of uncertainty to decision criteria underlying important large-scale and real time control of systems which rely on expensive simulation campaigns. Think of synthesis between off-line simulation of urban road traffic and ride demand with on-line measurements from potential riders and their routes in a rideshare pool. Or similarly the combination of limited data on disease spread combined with social-network backed simulation of epidemiological dynamics and entertainment of intervention strategies such as vaccination and quarantine. The talks will be on these methodologies and applied in those challenging modeling and optimization real-world problems.
14:00
From Parameter and Uncertainty Estimation to Optimal Experimental Design: Challenges in Biological Dynamical Systems Inference
Matthias Chung | Technical University of Berlin | Germany
Show details
Author:
Matthias Chung | Technical University of Berlin | Germany
Inference through data and mathematical modeling is particularly challenging for biological systems with noisy data, model uncertainties, and unknown mechanisms. Here, parameter and uncertainty estimation problems are typically ill-posed, meaning solutions do not exist, are not unique, or do not depend continuously on the data. Furthermore, experimentalists face the dilemma between accuracy and costs of an experiment. In this talk we will discuss new developments for parameter and uncertainty estimation for dynamical systems as well es discuss novel techniques for optimal experimental design.
14:30
- NEW - Sequential Learning of Active Subspaces
Nathan Wycoff | Virginia Tech | United States
Show details
Author:
Nathan Wycoff | Virginia Tech | United States
In recent years, active subspace methods (ASMs) have become a popular means of performing subspace sensitivity analysis on black-box functions. Naively applied, however, ASMs require gradient evaluations of the target function. In the event of noisy, expensive, or stochastic simulators, evaluating gradients via finite differencing may be infeasible. In such cases, often a surrogate model is employed, on which finite differencing is performed. When the surrogate model is a Gaussian process, we show that the ASM estimator is available in closed form, rendering the finite-difference approximation unnecessary. We use our closed-form solution to develop acquisition functions focused on sequential learning tailored to sensitivity analysis on top of ASMs. We also show that the traditional ASM estimator may be viewed as a method of moments estimator for a certain class of Gaussian processes. We demonstrate how uncertainty on Gaussian process hyperparameters may be propagated to uncertainty on the sensitivity analysis, allowing model-based confidence intervals on the active subspace. Our methodological developments are illustrated on several examples, and our open source code is available as an R package building on the CRAN package hetGP.
15:00
Multiobjective stochastic simulation optimization with correlation and heterogeneous noise
Sebastian Rojas Gonzalez | KU Leuven | Belgium
Show details
Authors:
Hamed Jalali | NEOMA Business School | France
Sebastian Rojas Gonzalez | KU Leuven | Belgium
We consider multi-objective simulation optimization problems with heteroscedastic noise, where we seek to find the non-dominated set of designs evaluated using noisy simulation evaluations. To perform the search of competitive designs, we propose a metamodel-based scalarization method, which explicitly characterizes both the extrinsic uncertainty of the unknown response surface, and the intrinsic uncertainty inherent in stochastic simulation. To differentiate the designs with the true best expected performance, we propose a multi-objective ranking and selection approach that accounts for correlation between the mean values of alternatives. Empirical results show that the proposed methods only require a small fraction of the available computational budget to find the optimal solutions.
15:30
Spatialised Generalized lambda distribution for risk-averse Bayesian Optimisation
Victor Picheny | Prowler.io | United Kingdom
Show details
Author:
Victor Picheny | Prowler.io | United Kingdom
We consider here the global optimisation of a stochastic black-box system, that is which objective function (output) given a set of decision parameters (inputs) is a random variable of unknown distribution. The standard approach is to optimize the function expectation and assume that the distribution around the mean is identical for all inputs (i.i.d. noise). In this work, we focus on the case where the shape as well as the amplitude of the distribution may change significantly within the input space. Further, we wish to design risk-averse strategies to account for the variability in the
objective. Our contribution is threefold: 1- We propose a new flexible surrogate model to provide an approximation of the entire distribution of the objective at any input point. To do so, we "spatialise" the so-called generalised lambda distribution, by modelling its parameters as latent Gaussian processes. Inference is made possible by the use of variational approaches through the "chained Gaussian process" framework. 2- We review the stochastic dominance concepts to compare distributions and characterise a set of incomparable inputs (i.e. optimal in the Pareto sense). 3- Using sequential Monte-Carlo concepts, we propose an algorithm to estimate this set while accounting for the GP uncertainties, and define a sequential sampling strategy based on this estimation. Our proposition is illustrated on several challenging toy problems.