First-Principle-Like Reinforcement Learning of Nonlinear Numerical Schemes for Conservation Laws

Abstract

In this study we present a universal nonlinear numerical scheme design method for nonlinear conservation laws, enabled by multi-agent reinforcement learning (MARL). Unlike contemporary approaches based on supervised learning or reinforcement learning, our method does not rely on reference data or empirical design. Instead, a first-principle-like approach using fundamental computational fluid dynamics (CFD) principles, including total variation diminishing (TVD) and k-exact reconstruction, is employed to design nonlinear numerical schemes. The third-order finite volume scheme is employed as the workhorse to test the performance of the MARL-based nonlinear numerical scheme design method. Numerical results demonstrate that the new MARL-based method can strike a balance between accuracy and numerical dissipation in nonlinear numerical scheme design, and outperforms the third-order MUSCL (Monotonic Upstream-centered Scheme for Conservation Laws) with the van Albada limiter for shock capturing. Furthermore, we demonstrate for the first time that a numerical scheme trained from one-dimensional (1D) Burgers’ equation simulations can be directly used for numerical simulations of both 1D and 2D (two-dimensional constructions using the tensor product operation) Euler equations. The framework of the MARL-based numerical scheme design concepts can incorporate, in general, all types of numerical schemes as simulation machines.