Despite the remarkable growth in computational power, it is still very computationally expensive to simulate most real-world systems in full detail, including a comprehensive analysis of parameter and model uncertainty. In such situations, data-driven approaches are tractable computational methods that provide useful empirical characterizations of uncertainty and have been successfully exploited in recent years. The increasing availability of very large data sets for this purpose makes techniques in machine learning an attractive toolbox for uncertainty emulation and characterization.
This minisymposium focuses on recent advances in uncertainty quantification algorithmic developments and applications based on data-driven and machine learning approaches in large-scale applications. Topics include data-driven surrogate construction, data assimilation, and physics-informed machine learning based on a limited number of data/observations and provide guidance for the system design, forecasting, etc.
14:00
Big Data Assimilation in Numerical Weather Prediction and Perspectives toward DA-UQ Collaboration
Takemasa Miyoshi | RIKEN | Japan
Show details
Author:
Takemasa Miyoshi | RIKEN | Japan
Data Assimilation (DA) was introduced in numerical weather prediction to combine computer model forecasts with real-world observations, using dynamical systems theory and statistical mathematics. Computing, sensing, and information/communication technologies are all advancing rapidly, and DA is becoming more popular as a means to perform cyber-physical fusion in broader sciences and technology fields. At RIKEN, the Japan’s flagship research institute for all sciences, we pioneered future possibilities of numerical weather prediction by taking advantage of the powerful K computer, and Big Data from advanced sensing technologies such as the Phased Array Weather Radar and the Himawari-8 geostationary satellite. We thus developed innovative “Big Data Assimilation” (BDA) technology, and made possible a 30-second-update of severe weather prediction at 100-m resolution, two orders of magnitude higher and faster than what is currently used in operational numerical weather prediction centers. DA aims to quantify probability distributions of dynamical model state variables and other parameters and has many things in common with Uncertainty Quantification which has been developed independently in applied mathematics. I will talk about our state-of-the-art research in numerical weather prediction and give a perspective towards DA-UQ collaboration.
14:30
- CANCELED - Random regularization for adversarial robustness of neural networks
Zuoqiang Shi | Tsinghua University | China
Show details
Author:
Zuoqiang Shi | Tsinghua University | China
Motivated by the connection between ResNets and convection equations, we built convection-diffusion model
to regularize the model and gain better adversarial robustness. Using the celebrated Feynman-Kac formula, high dimensional convection-diffusion model can be solved by solving stochastic differential equations with Brownian motion. This formulation guides to an ensemble ResNets framework which is demonstrated to be very robust to adversarial attacks.
15:00
Trainability and Data-dependent Initialization of Over-parameterized ReLU networks
Yeonjong Shin | Brown University | United States
Show details
Author:
Yeonjong Shin | Brown University | United States
In this talk, the trainability of ReLU neural networks will be discussed. A network is said to
be trainable if the number of active neurons is sufficiently large for a learning task. With this
notion, we refer the probability of a network being trainable as trainability. We show that the
trainability of ReLU networks is a necessary condition for the successful training and serves as an
upper bound of the successful training rates. In order to quantify the trainability, we study the
probability distribution of the number of active neurons at the initialization. In many applications, over-specified or over-parameterized neural networks are successfully employed and shown
to be trained effectively. With the notion of trainability, we show that over-parameterization
is both a necessary and a sufficient condition for minimizing the training loss. Furthermore,
we propose a data-dependent initialization method in the over-parameterized setting. Numerical examples are provided to demonstrate the effectiveness of the method and our theoretical
findings.
15:30
- CANCELED - Stochastic Training of Residual Networks: a Differential Equation Viewpoint
Qi Sun | Beijing International Center for Mathematical Research, Peking University | China
Show details
Author:
Qi Sun | Beijing International Center for Mathematical Research, Peking University | China
During the last few years, significant attention has been paid to the stochastic training of artificial neural networks. In this talk, residual networks with noise injection are regarded as weak approximations of stochastic differential equations. Such observation bridges the stochastic training processes with the optimal control of backward Kolmogorov's equations. This not only offers a novel perspective on the regularization effects but also sheds light on the design of stochastic training strategies.