Neural Q-learning for solving PDEs
Samuel N. Cohen, Deqing Jiang, Justin Sirignano; 24(236):1−49, 2023.
Abstract
Solving high-dimensional partial differential equations (PDEs) is a significant challenge in scientific computing. In this study, we propose a novel numerical method called the “Q-PDE” algorithm, which utilizes the Q-learning algorithm from reinforcement learning to solve elliptic-type PDEs. The advantage of our approach is that it is mesh-free, allowing it to potentially overcome the curse of dimensionality. By employing a neural tangent kernel (NTK) approach, we mathematically prove that the neural network approximator used in the Q-PDE algorithm converges to the trajectory of an infinite-dimensional ordinary differential equation (ODE) as the number of hidden units tends to infinity. For monotone PDEs, even in the absence of a spectral gap in the NTK, we further demonstrate that the limit neural network, which satisfies the infinite-dimensional ODE, strongly converges in the L^2 norm to the solution of the PDE as the training time tends to infinity. Moreover, we establish that any fixed point of the wide-network limit for the Q-PDE algorithm is a solution of the PDE, regardless of the monotone condition. Finally, we evaluate the numerical performance of the Q-PDE algorithm on various elliptic PDEs.
[abs]