This paper presents a machine learning framework called Graph Network-based Simulators (GNS) that can learn to simulate a wide variety of complex physical domains, including fluids, rigid solids, and deformable materials. The GNS framework represents the state of a physical system as a graph of particles and computes dynamics through learned message-passing. The model generalizes well from single-timestep predictions with thousands of particles during training to different initial conditions, thousands of timesteps, and at least an order of magnitude more particles at test time. The model is robust to hyperparameter choices across various evaluation metrics, with the main determinants of long-term performance being the number of message-passing steps and mitigating error accumulation by corrupting training data with noise. The GNS framework advances the state-of-the-art in learned physical simulation and holds promise for solving complex forward and inverse problems. The model is implemented in a single deep learning architecture and can accurately simulate a wide range of physical systems where fluids, rigid solids, and deformable materials interact. The model also generalizes well to much larger systems and longer time scales than those on which it was trained. The GNS model is compared to previous approaches and found to be simpler, more generally applicable, and more accurate. The model is tested on various physical domains, including water, sand, and goop, and shows strong performance in simulating these materials and their interactions. The model is also tested on generalization tasks, showing its ability to handle different initial conditions, object shapes, and larger domains. The GNS model is implemented using standard deep learning building blocks and uses a graph-based approach to simulate physical systems. The model's performance is evaluated using various metrics, including particle-wise MSE, optimal transport, and maximum mean discrepancy. The results show that the GNS model can accurately simulate complex physical systems and generalize well to new scenarios. The model is also compared to other approaches, such as CConv, and is found to be more accurate and generalizable. The GNS model is a powerful framework for learning to simulate complex physical systems and has the potential to be applied to a wide range of problems in science and engineering.This paper presents a machine learning framework called Graph Network-based Simulators (GNS) that can learn to simulate a wide variety of complex physical domains, including fluids, rigid solids, and deformable materials. The GNS framework represents the state of a physical system as a graph of particles and computes dynamics through learned message-passing. The model generalizes well from single-timestep predictions with thousands of particles during training to different initial conditions, thousands of timesteps, and at least an order of magnitude more particles at test time. The model is robust to hyperparameter choices across various evaluation metrics, with the main determinants of long-term performance being the number of message-passing steps and mitigating error accumulation by corrupting training data with noise. The GNS framework advances the state-of-the-art in learned physical simulation and holds promise for solving complex forward and inverse problems. The model is implemented in a single deep learning architecture and can accurately simulate a wide range of physical systems where fluids, rigid solids, and deformable materials interact. The model also generalizes well to much larger systems and longer time scales than those on which it was trained. The GNS model is compared to previous approaches and found to be simpler, more generally applicable, and more accurate. The model is tested on various physical domains, including water, sand, and goop, and shows strong performance in simulating these materials and their interactions. The model is also tested on generalization tasks, showing its ability to handle different initial conditions, object shapes, and larger domains. The GNS model is implemented using standard deep learning building blocks and uses a graph-based approach to simulate physical systems. The model's performance is evaluated using various metrics, including particle-wise MSE, optimal transport, and maximum mean discrepancy. The results show that the GNS model can accurately simulate complex physical systems and generalize well to new scenarios. The model is also compared to other approaches, such as CConv, and is found to be more accurate and generalizable. The GNS model is a powerful framework for learning to simulate complex physical systems and has the potential to be applied to a wide range of problems in science and engineering.