PhysDreamer is a physics-based approach that enables static 3D objects to interact dynamically by leveraging the object dynamics priors learned by video generation models. The method estimates physical material properties of static 3D objects to synthesize realistic responses to novel interactions, such as external forces or agent manipulations. PhysDreamer represents 3D objects using 3D Gaussians, models the physical material field with a neural field, and simulates 3D dynamics using the differentiable Material Point Method (MPM). The differentiable simulation and rendering allow for direct optimization of the physical material field and initial velocity field by matching pixel space observations. PhysDreamer is evaluated through a user study, demonstrating its effectiveness in generating realistic interactive motion. The results show that PhysDreamer significantly outperforms existing techniques in terms of motion realism, validating the effectiveness of leveraging video generation priors for estimating physical material properties and synthesizing interactive 3D dynamics. PhysDreamer enables static 3D objects to dynamically respond to interactive stimuli in a physically plausible manner, contributing to more engaging and realistic virtual experiences. The main contributions of the work include enabling static 3D objects to dynamically respond to interactive stimuli in a physically plausible manner and taking a step towards more engaging and realistic virtual experiences.PhysDreamer is a physics-based approach that enables static 3D objects to interact dynamically by leveraging the object dynamics priors learned by video generation models. The method estimates physical material properties of static 3D objects to synthesize realistic responses to novel interactions, such as external forces or agent manipulations. PhysDreamer represents 3D objects using 3D Gaussians, models the physical material field with a neural field, and simulates 3D dynamics using the differentiable Material Point Method (MPM). The differentiable simulation and rendering allow for direct optimization of the physical material field and initial velocity field by matching pixel space observations. PhysDreamer is evaluated through a user study, demonstrating its effectiveness in generating realistic interactive motion. The results show that PhysDreamer significantly outperforms existing techniques in terms of motion realism, validating the effectiveness of leveraging video generation priors for estimating physical material properties and synthesizing interactive 3D dynamics. PhysDreamer enables static 3D objects to dynamically respond to interactive stimuli in a physically plausible manner, contributing to more engaging and realistic virtual experiences. The main contributions of the work include enabling static 3D objects to dynamically respond to interactive stimuli in a physically plausible manner and taking a step towards more engaging and realistic virtual experiences.