February 27, 2024 | NIKOLA B. KOVACHKI, SAMUEL LANTHALER, AND ANDREW M. STUART
This paper reviews the field of operator learning, which involves approximating nonlinear operators mapping between Banach spaces of functions using machine learning techniques. The focus is primarily on neural operators, which leverage the success of deep neural networks in finite-dimensional settings. The paper discusses the motivation for viewing high-dimensional vectors as functions and the benefits of this approach in various applications. It also covers related work on learning linear operators, Gaussian processes, and random features.
The review begins with an introduction to operator learning, including the motivation for viewing high-dimensional vectors as functions and the importance of considering algorithms in the continuum limit before discretization. The literature review covers algorithms on function space, supervised learning on function space, and approximation theory. The paper then introduces the concept of operator learning as a supervised learning problem on Banach spaces, detailing the training and testing processes.
The paper discusses specific supervised learning architectures, including PCA-Net, DeepONet, Fourier Neural Operator (FNO), and random features methods. Each architecture is described in detail, highlighting their unique features and how they are trained and tested. The paper also explores the extraction of latent structure within these architectures and provides an example of fluid flow in a porous medium.
The review concludes with a discussion on universal approximation, proving that encoder-decoder-net architectures can approximate a wide class of operators to arbitrary accuracy. The universality of DeepONet and PCA-Net is established, along with extensions and variants of these architectures. The paper also discusses the universality of FNO and the general structure of neural operators, emphasizing the importance of integral kernels in their approximation capabilities.This paper reviews the field of operator learning, which involves approximating nonlinear operators mapping between Banach spaces of functions using machine learning techniques. The focus is primarily on neural operators, which leverage the success of deep neural networks in finite-dimensional settings. The paper discusses the motivation for viewing high-dimensional vectors as functions and the benefits of this approach in various applications. It also covers related work on learning linear operators, Gaussian processes, and random features.
The review begins with an introduction to operator learning, including the motivation for viewing high-dimensional vectors as functions and the importance of considering algorithms in the continuum limit before discretization. The literature review covers algorithms on function space, supervised learning on function space, and approximation theory. The paper then introduces the concept of operator learning as a supervised learning problem on Banach spaces, detailing the training and testing processes.
The paper discusses specific supervised learning architectures, including PCA-Net, DeepONet, Fourier Neural Operator (FNO), and random features methods. Each architecture is described in detail, highlighting their unique features and how they are trained and tested. The paper also explores the extraction of latent structure within these architectures and provides an example of fluid flow in a porous medium.
The review concludes with a discussion on universal approximation, proving that encoder-decoder-net architectures can approximate a wide class of operators to arbitrary accuracy. The universality of DeepONet and PCA-Net is established, along with extensions and variants of these architectures. The paper also discusses the universality of FNO and the general structure of neural operators, emphasizing the importance of integral kernels in their approximation capabilities.