**DiffTACTILE: A Physics-Based Differentiable Tactile Simulator for Contact-Rich Robotic Manipulation** **Authors:** Zilin Si, Gu Zhang, Qingwei Ben, Branden Romero, Zhou Xian, Chao Liu, Chuang Gan **Institutions:** CMU RI, Shanghai Jiao Tong University, Tsinghua University, MIT-CSAIL, MIT-IBM Watson AI Lab, UMass Amherst **Abstract:** DiffTACTILE is a physics-based differentiable tactile simulation system designed to enhance robotic manipulation with dense and physically accurate tactile feedback. Unlike prior tactile simulators that focus on rigid body manipulation and rely on simplified approximations, DiffTACTILE emphasizes physics-based contact modeling with high fidelity, supporting diverse contact modes and interactions with objects of various material properties. The system includes a Finite Element Method (FEM)-based soft body model for sensing elastomer, a multi-material simulator for modeling diverse object types, and a penalty-based contact model for handling contact dynamics. The differentiable nature of the system facilitates gradient-based optimization for refining physical properties in simulation using real-world data and efficient learning of tactile-assisted grasping and contact-rich manipulation skills. Additionally, a method is introduced to infer the optical response of the tactile sensor using a learning-based approach. DiffTACTILE is expected to serve as a useful platform for studying contact-rich manipulations, leveraging dense tactile feedback and differentiable physics. **Contributions:** - Introduces DiffTACTILE, a platform supporting various tactile-assisted manipulation tasks. - Models tactile sensors with FEM, objects with MLS-MPM, and cables with PBD. - Simulates contact between sensors and objects with a penalty-based contact model. - Accurately simulates the optical response of tactile sensors with high spatial variation. - Differentiable system enables efficient skill learning and system identification. - Evaluates on diverse manipulation tasks, including grasping, surface following, cable straightening, case opening, and object reposing. **Related Work:** - Discusses existing tactile simulation methods and their limitations. - Reviews differentiable physics-based simulation and its applications in robotics. **Experiments:** - Conducts system identification using real-world data to optimize simulator parameters. - Evaluates manipulation tasks with and without tactile feedback. - Compares gradient-based optimization with sampling-based and reinforcement learning approaches. **Conclusion:** DiffTACTILE advances skill learning for contact-rich robotic manipulation by providing models for tactile sensors, multi-material objects, and penalty-based contacts. The differentiability of the system aids in reducing the sim-to-real gap and improving skill learning efficiency. Future work includes integrating the simulator into common robotic simulation frameworks and exploring multi-modal learning in simulation.**DiffTACTILE: A Physics-Based Differentiable Tactile Simulator for Contact-Rich Robotic Manipulation** **Authors:** Zilin Si, Gu Zhang, Qingwei Ben, Branden Romero, Zhou Xian, Chao Liu, Chuang Gan **Institutions:** CMU RI, Shanghai Jiao Tong University, Tsinghua University, MIT-CSAIL, MIT-IBM Watson AI Lab, UMass Amherst **Abstract:** DiffTACTILE is a physics-based differentiable tactile simulation system designed to enhance robotic manipulation with dense and physically accurate tactile feedback. Unlike prior tactile simulators that focus on rigid body manipulation and rely on simplified approximations, DiffTACTILE emphasizes physics-based contact modeling with high fidelity, supporting diverse contact modes and interactions with objects of various material properties. The system includes a Finite Element Method (FEM)-based soft body model for sensing elastomer, a multi-material simulator for modeling diverse object types, and a penalty-based contact model for handling contact dynamics. The differentiable nature of the system facilitates gradient-based optimization for refining physical properties in simulation using real-world data and efficient learning of tactile-assisted grasping and contact-rich manipulation skills. Additionally, a method is introduced to infer the optical response of the tactile sensor using a learning-based approach. DiffTACTILE is expected to serve as a useful platform for studying contact-rich manipulations, leveraging dense tactile feedback and differentiable physics. **Contributions:** - Introduces DiffTACTILE, a platform supporting various tactile-assisted manipulation tasks. - Models tactile sensors with FEM, objects with MLS-MPM, and cables with PBD. - Simulates contact between sensors and objects with a penalty-based contact model. - Accurately simulates the optical response of tactile sensors with high spatial variation. - Differentiable system enables efficient skill learning and system identification. - Evaluates on diverse manipulation tasks, including grasping, surface following, cable straightening, case opening, and object reposing. **Related Work:** - Discusses existing tactile simulation methods and their limitations. - Reviews differentiable physics-based simulation and its applications in robotics. **Experiments:** - Conducts system identification using real-world data to optimize simulator parameters. - Evaluates manipulation tasks with and without tactile feedback. - Compares gradient-based optimization with sampling-based and reinforcement learning approaches. **Conclusion:** DiffTACTILE advances skill learning for contact-rich robotic manipulation by providing models for tactile sensors, multi-material objects, and penalty-based contacts. The differentiability of the system aids in reducing the sim-to-real gap and improving skill learning efficiency. Future work includes integrating the simulator into common robotic simulation frameworks and exploring multi-modal learning in simulation.