whi_3dobject_tracking

3D object tracking node leveraging the M3T from 3DObjectTracking

Citation

If you find our work useful, please cite us with:

@InProceedings{Stoiber_2023_IROS,
    author = {Stoiber, Manuel and Elsayed, Mariam and Rechert, Anne E. and Steidle, Florian and Lee, Dongheui and Triebel, Rudolph},
    title  = {Fusing Visual Appearance and Geometry for Multi-Modality 6DoF Object Tracking},
    year   = {2023},
}

Dependency

The WHI's msg interface:

cd <your workspace>/src
git clone https://github.com/xinjuezou-whi/whi_motion_interface.git

And the Intel librealsense2, please refer to its website

Build

Make sure your gcc version is later than 8 with command:

gcc -v

Otherwise run following commands to upgrade it:

sudo apt install gcc-9 g++-9
sudo update-alternatives --remove-all gcc 
sudo update-alternatives --remove-all g++
sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-9 9
sudo update-alternatives --install /usr/bin/g++ g++ /usr/bin/g++-9 9

Then build the package:

cd <your workspace>/src
git clone https://github.com/xinjuezou-whi/whi_3dobject_tracking.git
catkin build
source <your workspace>/devel/setup.bash

Publish

color_view(sensor_msgs::Image): the stream of color camera

depth_view(sensor_msgs::Image): the stream of depth camera

tcp_pose(whi_interfaces::WhiTcpPose): the estimated 6DOF pose

Service

tcp_pose(whi_interfaces::WhiSrvTcpPose): the estimated 6DOF pose

Config

whi_3dobject_tracking:
  image_resolution: [848, 480] # width, height
  frame_rates: [60, 60] # color and depth
  align_to_color: false
  view_color: true
  view_depth: true
  print_intrinsics: false
  visualize_pose_result: false
  use_region_modality: true
  use_depth_modality: true
  use_texture_modality: false
  measure_occlusions: false
  model_occlusions: false
  tracking_on_start: false
  bodies: ["io_485"] #[ "io_485", "triangle" ]
  directory: /home/whi/catkin_workspace/src/whi_3dobject_tracking/data
  color_topic: color_view
  color_overlay_topic: color_overlay_view
  depth_topic: depth_view
  depth_overlay_topic: depth_overlay_view
  pose_topic: tcp_pose
  pose_service: tcp_pose
  pose_frame: camera
  # chin: [0, 0, 0, -1.5708, 0, -1.5708] and [0.18, 0, 0]
  # world_to_tcp: [0, 0, 0, -1.5708, 0, -1.5708]
  # object_to_tcp: [0, 0, 0, 0, -1.5708, 0]
  # position_reference: [0.18, 0, 0]
  # ur: [0, 0, 0, 1.5708, 0, 0] and [0, -0.18, 0]
  world_to_tcp: [0, 0, 0, 1.5708, 0, 0]
  object_to_tcp: [0, 0, 0, 1.5708, -1.5708, 0]
  position_reference: [0, -0.18, 0]
  euler_muliplier: [0.0, 0.0, 0.0]