DIFFTACTILE is a physics-based differentiable tactile simulator designed to enhance robotic manipulation with dense and physically accurate tactile feedback. Unlike prior tactile simulators that focus on rigid bodies and simplified material models, DIFFTACTILE emphasizes physics-based contact modeling, supporting simulations of diverse contact modes and interactions with objects of varying material properties. The system includes a Finite Element Method (FEM)-based soft body model for tactile sensors, a multi-material simulator for various object types, and a penalty-based contact model for contact dynamics. The differentiable nature of DIFFTACTILE enables gradient-based optimization for refining physical properties and learning tactile-assisted grasping and contact-rich manipulation skills. Additionally, a method is introduced to infer the optical response of tactile sensors using a pixel-based neural module. DIFFTACTILE provides dense tactile feedback and differentiable physics, making it a useful platform for studying contact-rich manipulations. The system is implemented in Taichi, leveraging parallel GPU computing and auto-differentiation. The simulator is evaluated on diverse manipulation tasks, including grasping, surface following, cable straightening, case opening, and object reposing. The system is differentiable, allowing for system identification to reduce the sim-to-real gap. The simulator supports rigid, elastic, elastoplastic, and cable objects, and provides a data-efficient approach to simulate optical responses for vision-based tactile sensors. DIFFTACTILE outperforms existing tactile simulators in terms of system-wide differentiability, accuracy in modeling soft body dynamics and contact dynamics, and support for a wide range of object types. The system is evaluated on various tasks, demonstrating improved skill learning efficiency with tactile feedback. The simulator is compared with state-of-the-art tactile simulators, showing its effectiveness in contact-rich manipulation tasks. The system is implemented with Taichi, and the simulation pipeline includes pre-contact updates for FEM sensors and MPM objects, followed by two-way coupling to handle collision and calculate contact forces. The system is validated with real-world data, showing its ability to reduce the sim-to-real gap through gradient-based optimization. The optical simulation uses a data-driven approach to model the surface reflectance function, and the system is evaluated on various tasks, demonstrating its effectiveness in tactile-assisted manipulation. The system is also evaluated on grasping tasks, showing improved performance with tactile feedback. The system is compared with other tactile simulators, demonstrating its effectiveness in contact-rich manipulation tasks. The system is implemented with Taichi, and the simulation pipeline includes pre-contact updates for FEM sensors and MPM objects, followed by two-way coupling to handle collision and calculate contact forces. The system is validated with real-world data, showing its ability to reduce the sim-to-real gap through gradient-based optimization. The system is evaluated on various tasks, demonstrating its effectiveness in tactile-assisted manipulation.DIFFTACTILE is a physics-based differentiable tactile simulator designed to enhance robotic manipulation with dense and physically accurate tactile feedback. Unlike prior tactile simulators that focus on rigid bodies and simplified material models, DIFFTACTILE emphasizes physics-based contact modeling, supporting simulations of diverse contact modes and interactions with objects of varying material properties. The system includes a Finite Element Method (FEM)-based soft body model for tactile sensors, a multi-material simulator for various object types, and a penalty-based contact model for contact dynamics. The differentiable nature of DIFFTACTILE enables gradient-based optimization for refining physical properties and learning tactile-assisted grasping and contact-rich manipulation skills. Additionally, a method is introduced to infer the optical response of tactile sensors using a pixel-based neural module. DIFFTACTILE provides dense tactile feedback and differentiable physics, making it a useful platform for studying contact-rich manipulations. The system is implemented in Taichi, leveraging parallel GPU computing and auto-differentiation. The simulator is evaluated on diverse manipulation tasks, including grasping, surface following, cable straightening, case opening, and object reposing. The system is differentiable, allowing for system identification to reduce the sim-to-real gap. The simulator supports rigid, elastic, elastoplastic, and cable objects, and provides a data-efficient approach to simulate optical responses for vision-based tactile sensors. DIFFTACTILE outperforms existing tactile simulators in terms of system-wide differentiability, accuracy in modeling soft body dynamics and contact dynamics, and support for a wide range of object types. The system is evaluated on various tasks, demonstrating improved skill learning efficiency with tactile feedback. The simulator is compared with state-of-the-art tactile simulators, showing its effectiveness in contact-rich manipulation tasks. The system is implemented with Taichi, and the simulation pipeline includes pre-contact updates for FEM sensors and MPM objects, followed by two-way coupling to handle collision and calculate contact forces. The system is validated with real-world data, showing its ability to reduce the sim-to-real gap through gradient-based optimization. The optical simulation uses a data-driven approach to model the surface reflectance function, and the system is evaluated on various tasks, demonstrating its effectiveness in tactile-assisted manipulation. The system is also evaluated on grasping tasks, showing improved performance with tactile feedback. The system is compared with other tactile simulators, demonstrating its effectiveness in contact-rich manipulation tasks. The system is implemented with Taichi, and the simulation pipeline includes pre-contact updates for FEM sensors and MPM objects, followed by two-way coupling to handle collision and calculate contact forces. The system is validated with real-world data, showing its ability to reduce the sim-to-real gap through gradient-based optimization. The system is evaluated on various tasks, demonstrating its effectiveness in tactile-assisted manipulation.