Machine Learning (ML) has evolved into a core technology in many scientific applications. Solutions often require large labeled datasets to achieve high model accuracy. Unfortunately, this is a major bottleneck for many scientific computing applications, where numerical simulations are very expensive. Training on limited data can lead to significant uncertainties or errors when invoked outside the training space. But the fast execution of ML models once trained also make them ideal for exploring large numbers of runs for Uncertainty Quantification (UQ). Furthermore, many popular ML methods lack the needed mathematical support to prove robustness and reliability to motivate their use in scientific computing and uncertainty quantification UQ applications. This two-part mini-symposium will explore the interplay between ML and UQ, focusing in the following areas: (1) How do we leverage ML successes for scientific computing problems with uncertain inputs? (2) How do we use UQ methods to assess ML predictions and augment them with uncertainty estimates, error bounds, or prediction intervals? Addressing challenges in these areas will lead to greatly improve predictive capabilities. Methods that incorporate mathematical and scientific principles for uncertainty estimates in ML are needed. Literature in statistics can be leveraged for improving the model validation process and advances in UQ and V&V will greatly enhance the mathematical and scientific computing foundations for ML.
14:00
Learning-by-Calibrating: Improved Surrogate Models using Calibration as a Training Objective
Peer-Timo Bremer | Lawrence Livermore National Labs | United States
Show details
Authors:
Jayaraman J. Thiagarajan | Lawrence Livermore National Labs | United States
Rushil Anirudh | Lawrence Livermore National Labs | United States
Peer-Timo Bremer | Lawrence Livermore National Labs | United States
Data-driven machine learning (ML) methods are emerging as new foundations for evidence-based decision making and the future of scientific discovery. To fully realize their potential, we need to overcome significant hurdles in understanding the precision and uncertainty in purely data-driven predictions. Consequently, principled uncertainty quantification (UQ) methods are required to reliably assess their confidence in scenarios different from their training regime and verify that predictions arise from generalizable patterns rather than artifacts or biases in the training data. However, several existing methods for UQ in ML systems typically rely on black-box estimators, which are often difficult to validate and interpret. In this talk, we will present approaches that directly utilize confidence calibration as an optimization objective for producing prediction intervals in deep neural networks. We show that, under both Gaussian and more general non-Gaussian assumptions on the predictions, the calibration objective leads to both higher quality predictions as well as meaningful uncertainty estimates. Using data from a 1D semi-analytic numerical simulator for inertial confinement fusion, we show that the proposed approaches produce highly robust surrogate models.
14:30
Embracing unidentifiability in Bayesian model calibration with modularization
Kellin Rumsey | University of New Mexico | United States
Show details
Authors:
Kellin Rumsey | University of New Mexico | United States
Gabriel Huerta | Sandia National Laboratories | United States
Lauren Hund | Sandia National Laboratories | United States
Bayesian model calibration has become a powerful tool for the analysis of experimental data coupled with a physics-based mathematical model. The forward problem of prediction, especially within the range of data, is generally well-posed. However, there are many well-known issues with the approach when solving the inverse problem of parameter estimation, especially when the calibration parameters have physical interpretations. Unidentifiability is at the crux of these issues. In many practical applications, there are a small number of parameters which are considered ``of interest''. By focusing our efforts on these parameters and forfeiting the ability to learn about other parameters, robust inferential procedures can sometimes be obtained via a pseudo-Bayesian approach which is referred to as modularization.
In this article, we present modularization as a general estimation framework and provide a thorough discussion of when the framework should be used and, of equal importance, when it should not. We also develop an efficient algorithm for approximation of the modularization posterior numerically. Using this algorithm, modularization is applied to two example problems and the results are compared to the fully Bayesian approach. We show that modularization has many desirable statistical properties when the mathematical model is misspecified, at the cost of conservatism in parameter estimation.
15:00
Learning to regularize with a variational autoencoder for hydrologic inverse analysis
Daniel O’Malley | Los Alamos National Laboratory | United States
Show details
Authors:
Daniel O’Malley | Los Alamos National Laboratory | United States
John Golden | Los Alamos National Laboratory | United States
Inverse problems often involve matching observational data using a physical model that takes a large number of parameters as input. These problems tend to be under-constrained and require regularization to impose additional structure on the solution in parameter space. A difficulty in regularization is turning a complex conceptual model of this additional structure into a functional mathematical form. In this work we propose a method of regularization involving a machine learning technique known as a variational autoencoder (VAE). The VAE is trained to map a low-dimensional set of latent variables with a simple structure to the high-dimensional parameter space that has a complex structure. We train a VAE on unconditioned realizations of the parameters for a hydrological inverse problem. These unconditioned realizations neither rely on the observational data used to perform the inverse analysis nor require any forward runs of the physical model, thus making the computational cost of generating the training data minimal. The central benefit of this approach is that regularization is then performed on the latent variables from the VAE, which can be regularized simply. The VAE also reduces the number of variables in the optimization problem, thus making gradient-based optimization more computationally efficient. Our approach constitutes a novel framework for regularization and optimization, readily applicable to a wide range of inverse problems. We call the approach RegAE.
15:30
Data Driven Upscaling - Emulating Mesoscale Physics Using Machine Learning in Fractured Media
Nishant Panda | Los Alamos National Laboratory | United States
Show details
Authors:
Nishant Panda | Los Alamos National Laboratory | United States
Dave Osthus | Los Alamos National Laboratory | United States
Gowri Srinivasan | Los Alamos National Laboratory | United States
Daniel O’Malley | Los Alamos National Laboratory | United States
Viet Chau | Los Alamos National Laboratory | United States
Diane Oyen | Los Alamos National Laboratory | United States
Scale bridging is a critical need in computational sciences, where the modeling community has developed accurate physics models from first principles, of processes at lower length and time scales that influence the behavior at the higher scales of interest.
However, it is not computationally feasible to incorporate all of the lower length scale physics directly into upscaled models. This is an area where machine learning has shown promise, in building emulators of the lower length scale models which incur a mere fraction of the computational cost of the original higher fidelity models. We demonstrate the use of machine learning using an example in materials science estimating continuum scale parameters by emulating, with uncertainties, complicated mesoscale physics. We describe a new framework to emulate the fine scale physics, especially in the presence of micro-structures, using machine learning and showcase its usefulness by providing an example from modeling fracture propagation.