dexsuite/dex-retargeting

Dex Retargeting

Various retargeting optimizers to translate human hand motion to robot hand motion.

This repo originates from AnyTeleop Project

Installation

pip install dex_retargeting

To run the example, you may need additional dependencies for rendering and hand pose detection.

git clone https://github.com/dexsuite/dex-retargeting
cd dex-retargeting
pip install -e ".[example]"

Examples

Retargeting from human hand video

This type of retargeting can be used for applications like teleoperation, e.g. AnyTeleop.

Tutorial on retargeting from human hand video

Retarget from hand object pose dataset

This type of retargeting can be used post-process human data for robot imitation, e.g. DexMV.

Tutorial on retargeting from hand-object pose dataset

Citation

This repository is derived from the AnyTeleop Project and is subject to ongoing enhancements. If you utilize this work, please cite it as follows:

@inproceedings{qin2023anyteleop,
  title     = {AnyTeleop: A General Vision-Based Dexterous Robot Arm-Hand Teleoperation System},
  author    = {Qin, Yuzhe and Yang, Wei and Huang, Binghao and Van Wyk, Karl and Su, Hao and Wang, Xiaolong and Chao, Yu-Wei and Fox, Dieter},
  booktitle = {Robotics: Science and Systems},
  year      = {2023}
}

Acknowledgments

The robot hand models in this repository are sourced directly from dex-urdf. The robot kinematics in this repo are based on pinocchio. Examples use SAPIEN for rendering and visualization.

The PositionOptimizer leverages methodologies from our earlier project, From One Hand to Multiple Hands. Additionally, the DexPilotOptimizeris crafted using insights from DexPilot.