DEPRECATED AND WILL BE PHASED OUT IN FAVOR OF THE GDR-NET PIPELINE. DOCKER IMAGES CAN'T BE BUILD ANYMORE

Verefine Pipeline

Pipeline for detecting objects and estimating and refining their pose.

The pipeline is implemented to use YCB-V objects and configured for the use on our Toyota HSR Sasha.

Startup using the compose file(s)

The following commands will download the necessary data and then build all the docker containers and start them.

If the containers were already built before, you can still use the same commands (except download_data.sh) to start the pipeline.

densefusion + verefine:

./download_data.sh
cd compose/densefusion_pipeline
xhost +
docker-compose up

Three Docker containers will be started:

maskrcnn: Mask-RCNN trained on YCB-V Dataset
verefine: Pose Estimation with DenseFusion and refinement using VeREFINE
hsr-grasping: Node that calls detect, estimate_refine service and delivers object poses

ppf + verefine:

./download_data.sh
cd compose/densefusion_pipeline
xhost +
docker-compose up

Three Docker containers will be started:

maskrcnn: Mask-RCNN trained on YCB-V Dataset
ppf: Pose Estimation with PPF and refinement using VeREFINE
hsr-grasping: Node that calls detect, estimate_refine service and delivers object poses

Build Dockerfile with global context

maskrcnn: docker build -t maskrcnn -f src/maskrcnn/Dockerfile .

hsr-grasping: docker build -t hsr-grasping -f src/task/Dockerfile .

verefine: docker build -t verefine -f src/task/Dockerfile .

ppf: docker build -t verefine -f src/ppf/Dockerfile .

Visualization

In RVIZ you can view the final refined poses and the object segmentation by Mask-RCNN. They are published as images to these topics:

/hsr_grasping/segmentation
/verefine/estimated_poses (densefusion)
/ppf_estimation_and_verefine_refinement/verefine_debug (ppf)

Service

The pipeline will advertise a service /hsr_grasping/get_poses of the type get_poses.srv. The response represents the refined poses of the detected objects in the camera frame.

The services that are internally called are

/hsr_grasping/detect_objects of the type detectron2_service_server.srv
/hsr_grasping/estimate_poses of the type [estimate_poses.srv](https://github.com/v4r-

Main Service

get_poses.srv

---
PoseWithConfidence[] poses

Internal Services

detectron2_service_server.srv

sensor_msgs/Image image
---
object_detector_msgs/Detections detections

estimate_poses.srv

Detection det
sensor_msgs/Image rgb
sensor_msgs/Image depth
---
PoseWithConfidence[] poses

Important Messages

PoseWithConfidence.msg

string name
geometry_msgs/Pose pose
float32 confidence

Detecions.msg

Header header

uint32 height
uint32 width

Detection[] detections

Detection.msg

string name
float32 score
BoundingBox bbox
int64[] mask

Configurations

Config-Files

The params for camera, model paths and topics are in config/params.yaml Change this according to your project

im_width  # input image widht
im_height: # input image height
intrinsics:
- [538.391033533567, 0.0, 315.3074696331638]  # camera intrinsics
- [0.0, 538.085452058436, 233.0483557773859]
- [0.0, 0.0, 1.0]  

cfg_dir: "/verefine/data"  # config directory to load ppf_pose_estimation_config.ini for ppf
models_dir: "/verefine/data/models/renderer"  # ppf model directory

ycbv_names: /verefine/data/ycbv_names.json  # ycbv mapping form *.ply to object name
ycbv_verefine: /verefine/data/ycbv_verefine.json  # necessary verefine information about the models

color_topic: /hsrb/head_rgbd_sensor/rgb/image_rect_color #  rgb image topic
depth_topic: /hsrb/head_rgbd_sensor/depth_registered/image_rect_raw  # depth image topic
camera_info_topic: /hsrb/head_rgbd_sensor/rgb/camera_info # camera info topic

debug_visualization: True # for ppf 
publish_tf: True
max_dist: 1.0  # crop points farther than this [meters]
down_scale: 1  # down_scale speeds things up by already subsampling the image [power of 2]

distance_threshold: 0.01 # for plane detection
to_meters: 0.001 # for plane detection

ROS Master

The ROS Master is set in the docker-compose.yml file for each container

environment:
      ROS_MASTER_URI: "http://hsrb:11311"

ROS Namespace

The Namespace is also defined in the docker-compose.yml file for each container. It is passed as command with the python script calls like this:

command: bash -c "source /maskrcnn/catkin_ws/devel/setup.bash; ROS_NAMESPACE=hsr_grasping python3 /maskrcnn/src/maskrcnn/ros_detection.py"

If you change it, the service names and visualization topics will change accordingly.

v4r-tuwien/verefine_pipeline