Over the past two decades, we've witnessed two revolutions in applied mathematics and high-dimensional approximation: the rise of sparse reconstruction techniques driven by compressed sensing, and a transformation in data science driven by machine learning with deep neural networks, a.k.a, deep learning. The former seeks to find a compressible representation of a given target function or signal, exploiting structure such as sparsity, parametric smoothness, or low-dimensionality of the solution manifold. The latter seeks to construct a nonlinear approximation from a given dataset, which generalizes well on unseen data points, through a series of compositions of affine and nonlinear mappings. This minisymposium highlights connections between these two topics, with particular attention to recent advances in the theory and algorithms in both approaches, as applied to problems in uncertainty quantification. By bringing together researchers from these two emerging fields, we hope to foster discussion and collaboration on novel theoretical and computational advances in sparse approximation and deep learning, leading to new directions for research.
14:00
Greedy algorithms for sparse high-dimensional function approximation
Simone Brugiapaglia | Concordia University | Canada
Show details
Author:
Simone Brugiapaglia | Concordia University | Canada
In the context of high-dimensional function approximation from pointwise samples, sparse recovery techniques based on compressed sensing are know to lessen the curse of dimensionality with respect to the sampling complexity. Indeed, in the case of d-dimensional tensor product domains and using orthogonal polynomial expansions, the number of samples sufficient to recover the best s-term approximation rate is known to scale only polylogarithmically with respect to d. Yet, the corresponding recovery techniques based on weighted l1 minimization still suffer from the curse of dimensionality with respect to the reconstruction cost. The aim of this talk is to show the potential of greedy recovery algorithms based on orthogonal matching pursuit for tackling this challenge.
14:30
Deep ReLU neural networks and finite element spaces
Philipp Petersen | University of Vienna | Austria
Show details
Authors:
Philipp Petersen | University of Vienna | Austria
Joost Opschoor | ETH Zürich | Switzerland
Christoph Schwab | ETH Zürich | Switzerland
Carlo Marcati | ETH Zürich | Switzerland
We will analyse re-approximation of finite element spaces by deep neural networks with ReLU activation functions.
We will study the required complexity of these networks to emulate finite element spaces with varying mesh size and varying polynomial degree. In this framework, we will establish approximation results, in Sobolev norms, of smooth functions, Besov regular functions and also analytic, weighted analytic and Gevrey regular functions that frequently appear as solutions of elliptic problems with constant coefficients, but also as solutions of Navier-Stokes equations in polygons.
15:00
Solving Parametric PDEs with Deep Neural Networks: A Theoretical and Numerical Analysis
Mones Raslan | Technical University of Berlin | Germany
Show details
Authors:
Mones Raslan | Technical University of Berlin | Germany
Moritz Geist | Technical University of Berlin | Germany
Gitta Kutyniok | Technical University of Berlin | Germany
Philipp Petersen | University of Vienna | Austria
Reinhold Schneider | Technical University of Berlin | Germany
High-dimensional parametric partial differential equations (PPDEs) play an important role within the field of uncertainty quantification. In many cases, the set of all admissible solutions associated with the parameter space is inherently low dimensional. This fact forms the foundation for the reduced basis method. Recent numerical experiments by various researchers demonstrated the remarkable efficiency of using deep neural networks to solve parametric problems. In this talk, we start by presenting an approximation theoretical justification for this class of approaches. More precisely, we derive upper bounds on the complexity of ReLU neural networks approximating the solution maps of PPDEs. Without any knowledge of its concrete shape, we use the low-dimensionality of the solution manifold to obtain approximation rates which are significantly superior to those provided by classical approximation results. We use this low-dimensionality to guarantee the existence of a reduced basis. Then, for a large variety of PPDEs, we construct neural networks that yield approximations of the parametric maps essentially only depending on the size of the reduced basis. We conclude this talk with a numerical analysis which underlines the hypothesis that trained deep neural networks are indeed able to efficiently detect the solution manifold.
15:30
- NEW - Full strong error analysis for the training of deep neural networks with stochastic gradient descent
Timo Welti | ETH Zurich | Switzerland
Show details
Author:
Timo Welti | ETH Zurich | Switzerland
In this talk we present a full error analysis in the probabilistically strong sense of deep learning based empirical risk minimisation where the underlying deep artificial neural networks are trained using the stochastic gradient descent (SGD) optimisation method. The analysis is based on decomposing the overall error into three different parts (the approximation error, the generalisation error, and the optimisation error) and deriving strong error estimates for each of these separately. The convergence speed we obtain is rather slow, presumably far from optimal, and suffers under the curse of dimensionality. To the best of our knowledge, it is, however, the first full error analysis in the scientific literature of a deep learning algorithm where SGD is the employed optimisation method and, moreover, the first full error analysis in the scientific literature of a deep learning algorithm in the probabilistically strong sense.