Abstracts

Prof. Gianluigi Rozza
SISSA -International School for Advanced Studies -, Mathematics Area, mathLab, Trieste, Italy

Reduced Order Methods for PDEs: state of the art and perspectives with CFD applications in Industry, Medicine and Environmental Sciences

We provide the state of the art of Reduced Order Methods (ROM) for parametric Partial Differential Equations (PDEs), and we focus on some perspectives in their current trends and developments, with a special interest in parametric problems arising in offline-online Computational Fluid Dynamics (CFD). We focus on intrusive and then non-intrusive approaches.

Efficient parametrizations (random inputs, geometry, physics) are very important to be able to properly address an offline-online decoupling of the computational procedures and to allow competitive computational performances. Current ROM developments in CFD include: a better use of stable high fidelity methods, considering also spectral element method and finite volume discretizations, to enhance the quality of the reduced model too; more efficient sampling techniques to reduce the number of the basis functions, retained as snapshots, as well as the dimension of online systems; the improvements of the certification of accuracy based on residual based error bounds and of the stability factors, as well as the guarantee of the stability of the approximation with proper space enrichments, such as supremizers. For nonlinear systems, also the investigation on bifurcations of parametric solutions are crucial and they may be obtained thanks to a reduced eigenvalue analysis of the linearised operator. All the previous aspects are quite important in CFD problems to focus in real time on complex parametric industrial, environmental and biomedical flow problems, or even in a control flow setting, even in a multi-physics setting.

Model flow problems will focus on few benchmarks, as well as on simple fluid-structure interaction problems. Further examples of applications will be delivered concerning shape optimisation applied to industrial problems.

References:
Hesthaven, Rozza, Stamm - Certified Reduced Basis Methods for Parametrized Partial Differential Equations, Springer Briefs in Mathemathics, 2015.
Rozza, Huynh, Patera - Archives Computational Methods Engineering, ARCME, 2008, vol. 15, 229-275
Patera, Rozza - RB book online, augustine.mit.edu, 2007
Quarteroni, Rozza, Manzoni - Mathematics in Industry, 1:3, 2011
Chinesta, Huerta, Rozza, Willcox - Encyclopedia Comp. Mechanics, Wiley, 2015

More
Software: mathlab.sissa.it/cse-software
web: people.sissa.it/grozza

Karen Veroy-Grepl
RWTH, Aachen University

Introduction to Reduced Basis Methods: Theory and Applications

This two-part course aims to provide an introduction to the theory and applications of model order reduction (MOR) techniques for parametrized partial differential equations, with a particular focus on reduced basis methods.  

Introduction to Reduced Basis Methods

This first part begins with the basic aims of MOR for parametrized systems and an introduction to the basic elements of reduced basis (RB) methods for different types of parametrized PDEs. We show how the reduced basis method (1) constructs low order approximations by taking the Galerkin projection onto a space spanned by snapshots of the solution, thus essentially exploiting the low-dimensionality induced by the parametric dependence of the problem; (2) computes error estimates (and in some cases bounds)  to quantify the error with respect to the exact (finite element) solution; (3) systematically chooses the snapshots via a (weak) greedy algorithm; and (4) decomposes the computation in an offline-online procedure, thus leading to a significant reduction in the online cost to compute the approximation and error estimates.   We initially address the simplest case of elliptic and coercive problems, and then show how the basic methods can be extended to more complex problems.  

Application to Optimal Control, Data Assimilation, Optimal Experimental Design, and Problems in Industry

In the second part, we focus on the application of RB methods to different settings, particularly for optimal control, variational data assimilation, and optimal experimental design.  
In order to approximate the state of a physical system, data from physical measurements can be incorporated into a mathematical model to improve the state prediction. Discrepancies between data and models arise, since on the one hand, measurements are subject to errors and, onthe other hand, a model can only approximate the actual physical phenomenon. We show here how the reduced basis method can be applied to the 3D- and 4D-VAR methods of variational data assimilation for parametrized partial differential equations. The classical 3D- and 4DVAR methods make informed perturbations in order to find a state closer to the observations while main physical laws described by the model are maintained. We take inspiration from recent developments in state and parameter estimation and analyse the influence of the measurement space on the amplification of noise, and show how the reduced basis method can be used to aid in the selection of measurements.  To conclude, we show how some of the presented methods can be gainfully applied to problems in industry, particularly to problems in the geosciences and treatment planning of cancer ablation procedures.  

Carsten Carstensen
Humboldt Universität zu Berlin

Adaptive Finite Element Methods with Collective Marking

The standard adaptive finite element method with a marking strategy due to W.~Dörfler  is discussed on an abstract level with general error estimators and distances for the newest-vertex bisection. Solely four properties of those error estimators and distances, named the  axioms of adaptivity,  guarantee the convergence with optimal rates in terms of the error estimators: Stability (A1), reduction (A2), discrete reliability (A3), and quasi-orthogonality (A4). This general framework covers a huge  part of the existing literature on optimal  rates of adaptive schemes and is exemplified for the 2D Poisson model problem on polygonal domains  for conforming, nonconforming, and mixed finite element methods.

References:
C.~Carstensen, M.~Feischl, M.~Page, D.~Praetorius: Axioms of adaptivity.
Computer & Mathematics with Applications 67 (2014) 1195-1253 

R.~Stevenson: The completion of locally refined simplicial partitions created by bisection.
Math. Comp., 77 (261) (2008) 227-241

Adaptive Finite Element Methods with Separate Marking

The four axioms of adaptivity typically apply to examples of error estimators with a positive power of the mesh-size to deduce the reduction property (A2). The error analysis of mixed finite element schemes in the norm of H(div), for instance, includes the L2 norm of the data approximation error, which does not satisfy (A2). The data approximation of those terms also arises in the least squares finite element methods and led to a a separate marking for data resolution.  The presentation will focus on the example of a a least squares finite element method for the 2D Poisson model problem on polygonal domains with Raviart-Thomas and Courant finite elements. The extended version of the axioms of adaptivity lead to optimal convergence rates for an alternative error estimator.

References:
Carstensen and E.-J. Park: Convergence and optimality of adaptive least squares finite element methods.
SIAM J. Numer. Anal. 53 (2015) 43-62

Carstensen and H. Rabus: Axioms of adaptivity for separate marking for data resolution. 
SIAM J. Numer. Anal. 56 (2018) 2644-2665

Emmanuil Georgoulis
University of Leicester/ NTU Athens

 Adaptivity and a posteriori error estimation for Galerkin approximations of evolutionary problems.

Lecture 1: Reconstruction of Galerkin solutions as a tool of proving a posteriori error bounds.

The standard, classical, a priori error analysis of finite element, or more general of Galerkin-type, methods rests on two principles: the consistency and the stability of the method. The stability of the method in the Galerkin setting is typically shown via energy-type arguments. The resulting bounds are not computable as they involve the unknown exact solution. Far more useful in practice, however, are the so-called a posteriori error bounds, which involve the computed approximation to the problem at hand. These are typically proven using the stability properties of the exact PDE problem instead. To insert, however, the computed approximation into the PDE problem we typically need to reconstruct it to regain the regularity properties required by the PDE problem, e.g., continuity in time. In this part, I will present concepts of reconstruction for temporal and spatial discretisations, and I will show how to use these to prove practical a posteriori error bounds for Galerkin approximations of linear evolution PDEs in various norms.

Lecture 2: Adaptive Galerkin methods for evolutionary problems

Continuing from Lecture 1, I will move on to nonlinear evolution PDE problems, showing in particular that the use of reconstructions can be crucial in many challenging settings, such as finite time blow ups, robustness with respect to singular perturbations, and others. Moreover, I plan to highlight how to use the derived a posteriori error bounds within adaptive algorithms aiming to reduce the computational complexity of obtaining these approximations without any detrimental effect in the accuracy, especially in 3 spatial dimensions.

References (in preferred order of reading):

[1] C. Makridakis and R. H. Nochetto, Elliptic reconstruction and a posteriori error estimates for parabolic problems, SIAM J. Numer. Anal., 41, pp. 1585–1594 (2003).
[2] G. Akrivis, C. Makridakis, and R.H. Nochetto, A posteriori error estimates for the Crank-Nicolson method for parabolic equations, Math. Comp., 75, pp. 511–531 (2006). 
[3] C. Makridakis and R. H. Nochetto, A posteriori error analysis for higher order dissipative methods for evolution problems, Numer. Math. 104, no. 4, 489–514 (2006).
[4] E. H. Georgoulis, O. Lakkis, C. Makridakis and J. Virtanen. A posteriori error estimates for leap-frog and cosine methods for second order evolution problems. SIAM Journal on Numerical Analysis 54(1), pp. 120--136 (2016).
[5] E. H. Georgoulis, O. Lakkis and J.M. Virtanen. A posteriori error control for discontinuous Galerkin methods for parabolic problems. SIAM Journal on Numerical Analysis 49(2) pp. 427-458 (2011).
[6] A. Cangiani, E. H. Georgoulis, I. Kyza and S. Metcalfe. Adaptivity and blow-up detection for nonlinear evolution problems. SIAM Journal on Scientific Computing 38(6) pp. A3833–A3856 (2016).
[7] A. Cangiani, E. H. Georgoulis, A. Yu. Morozov, and O. J. Sutton. Revealing new dynamical patterns in a reaction-diffusion model with cyclic competition via a novel computational framework. Proceedings of the Royal Society A 474(2213) (2018).

Gitta Kutyniok
TU Berlin

Lecture 1; Solving Mathematical Problems by Deep Learning: Inverse Problems

Inverse problems in imaging such as denoising, recovery of missing data, or the inverse scattering problem appear in numerous applications. However, due to their increasing complexity, model-based methods are often today not sufficient anymore.  At the same time, we witness the
tremendous success of data-based methodologies, in particular, deep neural networks for such problems. However, at the same time, pure deep learning approaches often neglect known and valuable information from the modeling world.
In this talk we will start with an introduction into deep learning. Then we will focus on inverse problems, and review the key numerical approaches to solve such problems with a particular focus on deep learning-based methodologies. In a second part we will discuss one hybrid approach in more detail. For this, we will focus on the inverse problem of computed tomography, where one of the key issues is the limited angle problem. For this severely ill-posed inverse problem, we will develop a solver by combining the model-based method of sparse regularization by shearlets with the data-driven method of deep learning. This approach is faithful in the sense that we only learn the part which cannot be handled by model-based methods, while applying the theoretically controllable sparse regularization technique to all other parts. We further show that this algorithm
significantly outperforms previous methodologies, including methods entirely based on deep learning.

Lecture 2;  Solving Mathematical Problems by Deep Learning:  Partial Differential Equations

While the area of  inverse problems, predominantly from imaging, was very quick to embrace deep learning methods with tremendous success, the area of numerical analysis of partial differential
equations was much slowed. The reason might be the fact that no physical model for images exists, consequently making data-driven methods very effective, whereas rigorous physical models certainly do exist for numerical solvers of partial differential equations. But lately, impressive success could also be reported for very high dimensional partial differential equations with even a precise theoretical analysis that such approaches can beat the curse of dimensionality.
In this talk, we will first present an overview of numerical methods for partial differential equations based on deep neural networks. In a second part, we will then delve into high-dimensional  parametric partial differential equations, which appear in various contexts including control and
optimization problems, inverse problems, risk assessment, and uncertainty quantification. In most such scenarios the set of all admissible solutions associated with the parameter space is inherently low dimensional. This fact forms the foundation for the so-called reduced basis method. But
recently, numerical experiments have demonstrated the remarkable efficiency of using deep neural networks to solve parametric problems. We will then present a theoretical justification for this class of approaches. More precisely, for a large variety of parametric PDEs, we construct neural
networks that yield approximations of the parametric maps not suffering  from a curse of dimensionality and essentially only depending on the size of the reduced basis.

Daan Pelt
Centrum Wiskunde en Informatica, Amsterdam

Machine learning for large scientific images

In recent years, machine learning has proved successful in many imaging problems in a wide variety of application fields. Typically, successful approaches use convolutional neural networks that are trained using large amounts of training data. In many scientific applications, however, it is impossible to obtain the large amounts of training data needed for accurate training. In addition, popular neural network architectures are often developed for application to relatively small photographic images, while scientific images are typically significantly larger and contain more challenging artifacts compared with photographs. Therefore, new machine learning methods that are specifically designed for scientific applications are needed. In recent work, such methods have shown promising results, for example by using more efficient network architectures, including knowledge about the image formation inside the network, and adapting the application-specific data acquisition to the use of machine learning.

In this talk, I will give an introduction to convolutional neural networks and describe the specific challenges of applying them to large scientific images. Furthermore, I will describe recent work to improve machine learning methods for scientific applications. This work includes the introduction of a new type of neural network, and new ways of acquiring data. Results will be compared with popular existing machine learning methods and more classical approaches. In particular, I will focus on the application of machine learning to improve the image quality and analysis of X-ray tomography images. The results show that machine learning can be effectively used in practice to significantly improve reconstruction quality and analysis, even with severely limited data.

References:
1. Pelt, D. M., & Sethian, J. A. (2018). A mixed-scale dense convolutional neural network for image analysis. Proceedings of the National Academy of Sciences, 115(2), 254-259.
2. Pelt, D. M., Batenburg, K. J., & Sethian, J. A. (2018). Improving tomographic reconstruction from limited data using mixed-scale dense convolutional neural networks. Journal of Imaging, 4(11), 128.
3. Hendriksen, A. A., Pelt, D. M., Palenstijn, W. J., Coban, S. B., & Batenburg, K. J. (2019). On-the-Fly Machine Learning for Improving Image Resolution in Tomography. Applied Sciences, 9(12), 2445.

Anastasia Borovykh
Imperial College, UK

Analytic expressions for the output evolution of a deep neural network during training

In this talk we will focus on gaining insight into the output evolution of a deep neural network during training through analytic expressions. We will then use these expressions to understand the effects of the hyperparameters of the optimization algorithm on the output and generalization capabilities of the network. We will start with discussing a previously obtained result which shows that under specific assumptions a deep neural network becomes equivalent to a linear model. In this case one can explicitly solve for the network output during training, and the effects of the training hyperparameters can be studied. For general deep networks the linear approximation is no longer sufficient and higher order approximations are required. Obtaining explicit expressions in this case if however no longer trivial. We present a Taylor-expansion based method to solve for higher-order approximations of the network output during training, and also in this case study the effects of the hyperparameters of the optimization algorithm on the network output and generalization capabilities.