The paper introduces a scalable approach to enforce hard physical constraints using Mixture-of-Experts (MoE) in neural networks. The authors address the challenge of imposing strict adherence to physical laws, such as conservation laws, during neural network training, which can improve accuracy, reliability, and convergence. Traditional methods often use soft constraints through loss function penalties, but these can lead to convergence issues and lack reliability at inference time. The proposed PI-HC-MoE framework leverages differentiable physics and optimization to enforce hard constraints by decomposing the spatiotemporal domain into smaller regions, each solved by an "expert" through differentiable optimization. This approach allows for parallelization and reduces computational complexity, leading to more stable training and faster convergence. The method is evaluated on two challenging non-linear problems: diffusion-sorption and turbulent Navier-Stokes, demonstrating significantly higher accuracy and efficiency compared to standard differentiable optimization and soft constraint methods. The code for the framework is released to facilitate reproducibility and further exploration.The paper introduces a scalable approach to enforce hard physical constraints using Mixture-of-Experts (MoE) in neural networks. The authors address the challenge of imposing strict adherence to physical laws, such as conservation laws, during neural network training, which can improve accuracy, reliability, and convergence. Traditional methods often use soft constraints through loss function penalties, but these can lead to convergence issues and lack reliability at inference time. The proposed PI-HC-MoE framework leverages differentiable physics and optimization to enforce hard constraints by decomposing the spatiotemporal domain into smaller regions, each solved by an "expert" through differentiable optimization. This approach allows for parallelization and reduces computational complexity, leading to more stable training and faster convergence. The method is evaluated on two challenging non-linear problems: diffusion-sorption and turbulent Navier-Stokes, demonstrating significantly higher accuracy and efficiency compared to standard differentiable optimization and soft constraint methods. The code for the framework is released to facilitate reproducibility and further exploration.