一周活动预告(11.28-12.4)

Multiscale phenomena significantly impact on the computation and modeling for scientific and engineering problems. Multiscale finite element methods are used to build reduced order computational models for multiscale problems. In this talk, I will present some ideas and challenges for the multiscale finite element methods.

我将介绍我们最近关于神经网络的工作及其在偏微分方程数值解中的应用。在上次的演讲中，我主要介绍了使用神经网络数值求解线性和非线性标量双曲守恒定律方程，这类问题的解具有不连续性。针对此类问题，我展示了基于神经网络的方法在自由度数量方面优于基于网格的方法。

本次报告将主要介绍自适应网络扩增 (ANE) 方法。 开发 ANE 方法是为了解决一个基本的、开放的问题，即如何自动设计一个最优的神经网络架构，以在规定的精度内逼近函数和偏微分方程的解。此外，为了训练产生的非凸优化问题，ANE 方法提供了一个获得良好初始化的自然过程。

Complex nonlinear interplays of multiple scales give rise to many interesting physical phenomena and pose major difficulties for the computer simulation of multiscale PDE models in areas such as reservoir simulation, high frequency scattering and turbulence modeling. In this work, we introduce a hierarchical transformer (HT-Net) scheme to efficiently learn the solution operator for multiscale PDEs. We construct a hierarchical architecture with scale adaptive interaction range, such that the features can be computed in a nested manner and with a controllable linear cost. Self-attentions over a hierarchy of levels can be used to encode and decode the multiscale solution space over all scale ranges. In addition, we adopt an empirical loss function to counteract the spectral bias of the neural network approximation for multiscale functions. In the numerical experiments, we demonstrate the superior performance of the HT-Net scheme compared with state-of-the-art (SOTA) methods for representative multiscale problems.

本报告将介绍一种张量神经网络以及基于张量神经网络的机器学习算法。基于张量积的形式来建立神经网格，如此可以设计使用固定积分点的积分方式，将具有指数量级复杂度的高维积分转换成多项式量级的张量积分计算。如此就可以进行高维空间中的直接内积计算而不需要借助Monte-Carlo过程。接下来我们可以利用张量神经网络设计求解高维偏微分方程和特征值问题的机器学习算法，为高维偏微分方程的求解带来了更多的自由度和可操作性。。报告中将介绍基于张量神经网络的机器学习算法在求解高维特征值问题和多体问题中的应用。

In this paper, we consider the accuracy-enhancement of discontinuous Galerkin (DG) methods for solving partial differential equations (PDEs) with high order spatial derivatives. It is well known that there are highly oscillatory errors for finite element approximations to PDEs that contain hidden superconvergence points. To exploit this information, a Smoothness-Increasing Accuracy-Conserving (SIAC) filter is used to create a superconvergence filtered solution. This is accomplished by convolving the DG approximation against a B-spline  kernel. Previous theoretical result about this technique concentrated on first-  and second-order equations. However, for linear higher order equations, Yan  and Shu numerically demonstrated that it is possible to improve the accuracy  order to $2k+1$ for local discontinuous Galerkin (LDG) solutions using the SIAC  filter. In this work, we firstly provide theoretical proof for this observation. Furthermore, we prove the accuracy order of the ultra-weak local discontinuous Galerkin (UWLDG) solution could be improved to $2k+2-m$ using the SIAC  filter, where $m=[\frac{n}{2}]$, $n$ is the order of PDEs. Finally, we computationally demonstrate that for nonlinear higher order PDEs, we can also obtain a superconvergence approximation with the accuracy order of $2k+1$ or   $2k+2-m$ by convolving the LDG solution and the UWLDG solution against the  SIAC filter, respectively.

This paper revisits the bandit problem in the Bayesian setting. The Bayesian approach formulates the bandit problem as an optimization problem, and the goal is to find the optimal policy which minimizes the Bayesian regret. One of the main challenges facing the Bayesian approach is that computation of the optimal policy is often intractable, especially when the length of the problem horizon or the number of arms is large. In this paper, we first show that under a suitable rescaling, the Bayesian bandit problem converges to a continuous Hamilton-Jacobi-Bellman (HJB) equation. The optimal policy for the limiting HJB equation can be explicitly obtained for several common bandit problems, and we give numerical methods to solve the HJB equation when an explicit solution is not available. Based on these results, we propose an approximate Bayes-optimal policy for solving Bayesian bandit problems with large horizons. Our method has the added benefit that its computational cost does not increase as the horizon increases.

   Deep neural networks are powerful tools for approximating functions, and they are applied to successfully solve various problems in many fields. In this talk, we propose a neural network-based numerical method to solve partial differential equations. In this new framework, the method is designed on weak formulations, and the unknown functions are approximated by deep neural networks and test functions can be chosen by different approaches, for instance, basis functions of finite element methods, neural networks, and so on. Because the spaces of trial function and test function are different, we name this new approach by Deep Petrov-Galerkin Method (DPGM). The resulted linear system is not necessarily to be symmetric and square, so the discretized problem is solved by a least-squares method. Take the Poisson problem as an example, mixed DPGMs based on several mixed formulations are proposed and studied as well. In addition, we apply the DPGM to solve two classical time-dependent problems based on the space-time approach, that is, the unknown function is approximated by a neural network, in which temporal variable and spatial variables are treated equally, and the initial conditions are regarded as boundary conditions for the space-time domain. Finally, several numerical examples are presented to show the performance of the DPGMs, and we observe that this new method outperforms traditional numerical methods in several aspects: compared to the finite element method and finite difference method, DPGM is much more accurate with respect to degrees of freedom; this method is mesh-free, and can be implemented easily; mixed DPGM has good flexibility to handle different boundary conditions; DPGM can solve the time-dependent problems by the space-time approach naturally and efficiently. The proposed deep Petrov-Galerkin method shows strong potential in the field of numerical methods for partial differential equations.

深度学习中高度非凸非线性优化问题由于维数灾难求解十分困难，发展准确、快速、实时的随机优化算法非常重要。本报告简要探讨矩阵情形的自然梯度法和流形上的自然梯度法，探索大规模复杂任务的算法基础。

In this talk, we will introduce a nonlocal Cahn-Hilliard equation and its applications in 3D printing and fingerprint restoration. A Crank-Nicolson method is proposed to discrete the nonlocal Cahn-Hilliard equation, which was developed for modeling microphase separation of diblock copolymers. We prove that our proposed scheme is unconditionally energy stable. Then we will present a robust and efficient fingerprint image restoration algorithm and surface pattern generation method for 3D printing. The proposed method has a merit that the pixel values in the damaged fingerprint domain can be obtained using the image information from the outside of the damaged fingerprint region. Restoration of fingerprint based on the adjacent pixel information can ensure the accuracy of fingerprint information with low computational cost. Computational experiments are presented to demonstrate the efficiency of the proposed method.

Motivated by applications, like modeling of thin structures immersed in a fluid, we develop a well posedness theory for Newtonian and some non-Newtonian fluids under singular forcing in Lipschitz domains, and in convex polytopes. The main idea, that allows us to deal with such forces is that we study the problem in suitably weighted Sobolev spaces. We develop an a priori approximation theory, which requires to develop the stability of the Stokes projector over weighted spaces. In the case that the forcing is a linear combination of Dirac deltas, we develop a posteriori error estimators for the stationary Stokes and Navier Stokes problems. We show that our estimators are reliable and locally efficient and illustrate their performance within an adaptive method. We briefly comment on work regarding the Bousinessq system. Numerical experiments illustrate and complement our theory.

This talk is based on a series of works in collaboration with: R. Duran (Argentina), E. Otarola and A. Allendes (Chile).

Traditionally, atomistic simulations of crystalline defects use fixed, free or periodic boundary conditions. However, these “classical” boundary conditions can perturb the motion of crystalline defects – in particular of extended defects, for instance, dislocations [1]—and thus require large simulation cells that are computationally very expensive. Possibly the earliest multiscale boundary condition that avoids those issues has been developed by Sinclair et al. [2] In the 1970s who coupled the atomistic problem to an infinite elasticity problem. However, this approach is difficult to implement considering the coupling of two problems that are solved rather differently, and has, to date, not yet reached the state of full maturity.

In this talk, I will discuss our recent efforts in developing these multiscale boundary conditions towards applications in materials science, in particular, for simulation long dislocations[3-5]. Another focus will be set on the choice of numerical solver for the coupled atomistic/elasticity problem, and how such solvers can be integrated into established molecular dynamics software packages in a non- or least-intrusive way. In addition, I will show how the boundary conditions could be extended to add plasticity to the elasticity domain using discrete dislocation dynamics.

References

[1] Szajewski, B.A., & Curtin, W.A. (2015). Model. Simul. Mater. Sci. Eng. 23(2), 025008.

[2] Sinclair, J.E. et al. (1978). J. Appl. Phys. 49(7), 3890-3897.

[3] Hodapp, M. et al. (2019). Comput. Methods Appl. Mech. Eng. 348, 1039-1075.

[4] Hodapp, M. (2021). SIAM Multiscale Model. Simul. 19(4), 1499-1537.

[5] Hodapp, M. (2022). Commun. Comput. Phys. 32, 671-714.

[6] Anciaux, G. et al. (2018). J. Mech. Phys. Solids 118, 152-171.

System of interacting particles that display a wide variety of collective behaviors are ubiquitous in science and engineering, such as self-propelled particles, flocking of birds, milling of fish. Modeling interacting particle systems by a system of differential equations plays an essential role in exploring how individual behavior engenders collective behaviors, which is one of the most fundamental and important problems in various disciplines. Although the recent theoretical and numerical study bring a flood of models that can reproduce many macroscopical qualitative collective patterns of the observed dynamics, the quantitative study towards matching the well-developed models to observational data is scarce. 

We consider the data-driven discovery of macroscopic particle models with latent interactions. We propose a learning approach that models the latent interactions as Gaussian processes, which provides an uncertainty-aware modeling of interacting particle systems. We introduce an operator-theoretic framework to provide a detailed analysis of recoverability conditions, and establish statistical optimality of the proposed approach. Numerical results on prototype systems and real data demonstrate the effectiveness of the proposed approach.

The focus of this talk is on the numerical analysis of reaction-subdiffusion equations with variable time step by taking the widely used L1 scheme for an example. For the stability analysis, the discrete complementary convolution (DCC) kernels

are introduced to prove the discrete fractional-type Gronwall inequality. For the convergence analysis, the goals are theoretically challenging because the numerical Caputo formula always has a form of discrete convolutional summation. In addition, the technique here is also extended the study of multi-step schemes such as BDF2 with variable time step for classical parabolic equations.

To construct a parallel approach for solving optimization problems with orthogonality constraints is usually regarded as an extremely difficult mission, due to the low scalability of the orthogonalization procedure. However, such demand is particularly huge in some application domains such as material computation. In this talk, we introduce several efficient algorithms including feasible and infeasible methods. Such methods have much lower computational complexity and can also benefit from parallel computing since they are full of BLAS3 operations. In the infeasible algorithm based on a modified augmented Lagrange method, the orthogonalization procedure is only invoked once as the last step. Consequently, the main parts of the proposed algorithms can be parallelized naturally. We establish global subsequence convergence results for our proposed algorithms. Worst-case complexity and local convergence rate are also studied under some mild assumptions. Numerical experiments, including tests under parallel environment, illustrate that our new algorithms attain good performances and high scalability in solving discretized Kohn--Sham total energy minimization problems.

In this work, uniformly unconditionally stable first and second order finite difference schemes are developed for kinetic transport equations in the diffusive scaling. We derived an approximate evolution equation for the macroscopic density, from the formal solution of the distribution function, which is then discretized by following characteristics for the transport part with a backward finite difference semi-Lagrangian approach, while the diffusive part is discretized implicitly. The resulting schemes can be shown to be asymptotic preserving (AP) in the diffusive limit. Uniformly unconditional stabilities are verified from a Fourier analysis. Numerical experiments, including high dimensional problems, have demonstrated the corresponding orders of accuracy both in space and in time, uniform stability, AP property, and good performances of our proposed approach.

Large-scale optimization is at the forefront of modern data science, scientific computing, and applied mathematics with areas of interest, including high-dimensional PDE, inverse problems, machine learning, etc. First-order methods are workhorses for large-scale optimization due to modest computational cost and simplicity of implementation. Nevertheless, these methods are often agnostic to the structural properties of the problem under consideration and suffer from slow convergence, being trapped in bad local minima, etc. Natural gradient descent is an acceleration technique in optimization that takes advantage of the problem’s geometric structure and preconditions the objective function’s gradient by a suitable "natural" metric. Hence parameter update directions correspond to the steepest descent on a corresponding "natural" manifold instead of the Euclidean parameter space rendering a parametrization invariant descent direction on that manifold. Despite its success in statistical inference and machine learning, the natural gradient descent method is far from a mainstream computational technique due to the computational complexity of calculating and inverting the preconditioning matrix. This work aims at a unified computational framework and streamlining the computation of a general natural gradient flow via the systematic application of efficient tools from numerical linear algebra. We obtain efficient and robust numerical methods for natural gradient flows without directly calculating, storing, or inverting the dense preconditioning matrix. We treat Euclidean, Wasserstein, Sobolev, and Fisher–Rao natural gradients in a single framework for a general loss function.