/CtRNet-robot-pose-estimation

PyTorch implementation of CtRNet https://arxiv.org/abs/2302.14332

Primary LanguageJupyter NotebookMIT LicenseMIT

Markerless Camera-to-Robot Pose Estimation via Self-supervised Sim-to-Real Transfer

PyTorch implementation of CtRNet https://arxiv.org/abs/2302.14332

Dependencies

Recommend set up the environment using Anaconda. Code is developed and tested on Ubuntu 20.04.

  • Python(3.8)
  • Numpy(1.22.4)
  • PyTorch(1.10.0)
  • torchvision(0.11.1)
  • pytorch3d(0.6.2)
  • Kornia(0.6.3)
  • Transforms3d(0.3.1)

More details see environment.yml.

Usage

  • See inference_single_frame.ipynb for example single frame inference.
  • We provide ROS node for CtRNet, which subscribes image and joint state topics and publishes robot pose.
python ros_node/panda_pose.py

Dataset

  1. DREAM dataset
  2. Baxter dataset

Weights

Weights for Panda and Baxter can be found here.

Videos

Using CtRNet for visual servoing experiment with moving camera.

video_1.mp4
video_2.mp4