29 May 2024 | Maximilian Herde1,* , Bogdan Raonić1,2,* , Tobias Rohner1, Roger Käppeli1, Roberto Molinaro1, Emmanuel de Bézenac1, and Siddhartha Mishra1,2
POSEIDON is a foundation model designed to learn the solution operators of Partial Differential Equations (PDEs). It is based on a multiscale operator transformer (scOT) with time-conditioned layer norms, enabling continuous-in-time evaluations. A novel training strategy leverages the semi-group property of time-dependent PDEs to scale up the training data. POSEIDON is pre-trained on a diverse, large-scale dataset of fluid dynamics equations and evaluated on 15 challenging downstream tasks, covering various PDE types and operators. The model demonstrates excellent performance, outperforming baselines significantly in terms of sample efficiency and accuracy. It also generalizes well to new physics not seen during pretraining. POSEIDON scales with respect to model and data size, both for pretraining and downstream tasks. The paper provides insights into the mechanisms by which POSEIDON learns effective representations during pretraining and generalizes to unseen PDEs. The model and datasets are open-sourced.POSEIDON is a foundation model designed to learn the solution operators of Partial Differential Equations (PDEs). It is based on a multiscale operator transformer (scOT) with time-conditioned layer norms, enabling continuous-in-time evaluations. A novel training strategy leverages the semi-group property of time-dependent PDEs to scale up the training data. POSEIDON is pre-trained on a diverse, large-scale dataset of fluid dynamics equations and evaluated on 15 challenging downstream tasks, covering various PDE types and operators. The model demonstrates excellent performance, outperforming baselines significantly in terms of sample efficiency and accuracy. It also generalizes well to new physics not seen during pretraining. POSEIDON scales with respect to model and data size, both for pretraining and downstream tasks. The paper provides insights into the mechanisms by which POSEIDON learns effective representations during pretraining and generalizes to unseen PDEs. The model and datasets are open-sourced.