Author:
Anthony Nouy | Centrale Nantes / LMJL | France
The approximation of high-dimensional functions is a typical task in computational science. Examples of such problems can be found in physics, machine learning and uncertainty quantification.
The approximation of a high-dimensional function requires the introduction of approximation tools (or model classes) that exploit some specific structures of functions. In this talk, we consider the model classes of functions in tree-based tensor format, or tree tensor networks [1]. These are particular cases of feed-forward deep neural networks with multilinear activation functions and an architecture given by a dimension partition tree.
After a presentation of these tools, we present some recent results on their approximation power. We discuss the importance of choosing a good tree and present stochastic algorithms for addressing this combinatorial problem. We then describe adaptive learning algorithms for the estimation of tree tensor networks, using either independent samples in a classical empirical risk minimization framework [2,3], or adaptively chosen samples for a direct estimation of the tensors using empirical principal component analysis of multivariate functions [4]. These algorithms are particularly suited for situations where only a small number of samples are available, a typical situation in uncertainty quantification problems for complex models.
References:
[1] A. Nouy. Low-rank methods for high-dimensional approximation and model order reduction. In P. Benner, A. Cohen, M. Ohlberger, and K. Willcox, editors, Model Reduction and Approximation: Theory and Algorithms. SIAM, Philadelphia, PA, 2017.
[2] E. Grelier, A. Nouy, and M. Chevreuil. Learning with tree-based tensor formats.
arXiv e-prints, arXiv:1811.04455, 2018.
[3] E. Grelier, A. Nouy, and R. Lebrun. Learning high-dimensional probability distributions using tree tensor networks. ArXiv e-prints, arXiv:1912.07913, 2019.
[4] A. Nouy. Higher-order principal component analysis for the approximation of tensors in tree-based low-rank formats. Numerische Mathematik, 141(3):743--789, 2019.