The paper presents a machine learning framework called "Graph Network-based Simulators" (GNS) that can simulate complex physical systems involving fluids, rigid solids, and deformable materials. GNS represents the state of a physical system using particles as nodes in a graph and computes dynamics through learned message-passing. The framework is implemented in a single deep learning architecture and demonstrates strong generalization capabilities, including handling larger systems, longer time scales, and different initial conditions. The model's performance is robust to hyperparameter choices, with key determinants being the number of message-passing steps and mitigating error accumulation through noisy training data. GNS advances the state-of-the-art in learned physical simulation and has potential applications in solving complex forward and inverse problems. The paper also includes experimental details, comparisons with related work, and discussions on architectural choices and generalization.The paper presents a machine learning framework called "Graph Network-based Simulators" (GNS) that can simulate complex physical systems involving fluids, rigid solids, and deformable materials. GNS represents the state of a physical system using particles as nodes in a graph and computes dynamics through learned message-passing. The framework is implemented in a single deep learning architecture and demonstrates strong generalization capabilities, including handling larger systems, longer time scales, and different initial conditions. The model's performance is robust to hyperparameter choices, with key determinants being the number of message-passing steps and mitigating error accumulation through noisy training data. GNS advances the state-of-the-art in learned physical simulation and has potential applications in solving complex forward and inverse problems. The paper also includes experimental details, comparisons with related work, and discussions on architectural choices and generalization.