ContactSDF: Signed Distance Functions as Multi-Contact Models for Dexterous Manipulation

Wen Yang, Wanxin Jin

Arizona State University

Overview: we propose ContactSDF, a method that uses signed distance functions (SDFs) to approximate multi-contact models, including both collision detection and time-stepping routines. ContactSDF achieves a closed-form state prediction and end-to-end differentiability, enabling efficient model learning and optimization for contact-rich manipulation.

Long take uncut:
learning continuous reorientation from scratch on hardware using ContactSDF

Evaluation with learned ContactSDF-MPC (50Hz)

Continuous counterclockwise reorientation

Continuous clockwise reorientation

How does ContactSDF work?

ContactSDF has two key components: (1) SDF-based collision detection, where robot and environment points query a geometric SDF that is constructed directly from the polytope representation of the object mesh or point cloud; and (2) SDF-based time-stepping prediction, where a velocity-space SDF is proposed to approximate the time-stepping of multi-contact dynamics based on the contact info obtained from the collision detection. With these two SDFs, the ContactSDF achieves a closed-form state prediction and end-to-end differentiability (collision detection routine included).

More Simulation Results

Three-Ball Manipulation

In the three-ball manipulation task, the system includes three balls (red, green, and blue), each of which has 3-DoF (moving XYZ directions) and is actuated by a low-level position controller, and a 6-DoF object laying on a frictional ground. The goal is that three balls move the object from an initial pose to a desired on-ground target pose.

Cube
Foambrick
Stick

Allegro On-palm Manipulation

We also demonstrate the ContactSDF model on a Allegro hand for on-palm object reorientation. The system includes a 6-DoF object laying on the palm of the Allegro hand and the fingers are actuated by a low-level joint PD controller. The goal is to reorient the object (solid) from an initial pose to a desired target pose (transparent). Click the tabs to check the manipulation results for different objects.

Cube
Foambrick
Stick

Acknowledgements

We would like to thank Zhixian Xie and Aravind Prakash Senthil for their help with hardware experiments and discussions.

BibTeX

@article{yang2024contactsdf,
      title={ContactSDF: Signed Distance Functions as Multi-Contact Models for Dexterous Manipulation},
      author={Yang, Wen and Jin, Wanxin},
      journal={arXiv preprint arXiv:2408.09612},
      year={2024}
    }