Tackling the Curse of Dimensionality with Physics-Informed Neural Networks
COM3 Level 2
SR21, COM3 02-60
Abstract:
The curse-of-dimensionality taxes computational resources heavily with exponentially increasing computational cost as the dimension increases. This poses great challenges in solving high-dimensional partial differential equations (PDEs), as Richard E. Bellman first pointed out over 60 years ago. While there has been some recent success in solving numerical PDEs in high dimensions, such computations are prohibitively expensive, and true scaling of general nonlinear PDEs to high dimensions has never been achieved.
We develop new methods of scaling up physics-informed neural networks (PINNs) to solve arbitrary high-dimensional and high-order PDEs. The first new method, called Stochastic Dimension Gradient Descent (SDGD), decomposes the gradient of PDEs' and PINNs' residual into pieces corresponding to different dimensions and randomly samples a subset of these dimensional pieces in each iteration of training PINNs.
Furthermore, inspired by the Hessian trace operator in second-order PDEs, we introduce Hutchinson Trace Estimation (HTE) to accelerate and scale up PINN. This approach views high-dimensional PDE operators as the trace of Hessian matrices, thereby employing the Hessian vector product to estimate the trace unbiasedly. We demonstrate how SDGD and HTE can be unified and their difference.
Then, with the recently developed high-dimensional PDE solvers, we conduct extensive experiments on Hamilton-Jacobi-Bellman, Fokker-Planck, and other nonlinear PDEs with nonlinear, anisotropic, and inseparable solutions. We demonstrate respective algorithms for various PDEs and scale up PINNs to 100,000 dimensions, whose training can be done in a few hours or even minutes.
We also demonstrate how to employ score-based PINN to solve high-dimensional Fokker-Planck and fractional equations, and integrate it with the aforementioned SDGD and HTE techniques to expedite training.
Overall, this research expands the applicability of PINN to general high-dimensional equations, offering a novel solution to the problem of the dimensionality curse in science and engineering