ROS node for real-time body parts detection

This is a ROS implementation of a light-weight RefineNet neural network trained on the body parts data set. The NN is capable of detecting human body parts and can differentiate between heads, torsos, upper arms, lower arms, upper legs, and lower legs with a mean intersection-over-union (IoU) score of 0.649 (Nek18).

This node is part of a larger project with the objective to enable object-independent human-to-robot handovers using robotic vision. The code for this project can be found here.

The node can be implemented as publisher, service, or action. See below for more information.

Input: RGB image: sensor_msgs/CompressedImage
Output: Mask (0 background, 1-6 body parts): sensor_msgs/CompressedImage

Example from a frontal perspective (full body):

Example from a lateral perspective (arm and hand only):

Getting Started

The code has been tested with Python 3.6.

Hardware Requirements

RGB camera (for this project a realsense D435 was used)
GPU >= 2 GB

Software Requirements

ATTENTION: This package requires the ROS operating system!

Python: see requirements.txt
ROS packages:

rospy
actionlib
sensor_msgs
cv_bridge
ros_numpy

Launch

The ROS package contains 3 launch files: publisher, service an action.

Publisher: Publishes a mask every time a new image is published by the camera.
Serivce: Returns a mask upon service call.
Action: Returns a mask upon client call.

The input/output is identical for a all three nodes:

Input: RGB image: sensor_msgs/CompressedImage
Output: Mask (0 background, 1-6 body parts): sensor_msgs/CompressedImage

Configuration

The initial configuration can be changed by adapting the bodyparts.yaml file:

Camera:

topic: Rostopic the publisher is subscribing to. Altering this configuration has no impact on the service and action nodes.

Interfaces:

topic: Rostopic the publisher node is publishing to.
service: Rosservice for interacting with the service node.
action: Rostopic for interacting with the action node.

Visualization: The visualization mode publishes a color-inverted copy (BGR) of the original RGB image with the background blacked out. Please be aware that turning on the visualization increases computing time and network utilization substantially.

topic: Rostopic the node is publishing to (visualization).
activated: Turn on/off visualization: use only keywords "True" or "False"

GPU:

gpu: ID of the GPU (only usable if more than one GPU is available).

Model:

model: Number of NN-layers (RefineNet): use keywords "50", "101" or "152" only.

Acknowledgments

The ROS node is powered by the pytorch implementation of DrSleep. For more information on RefineNet please refer to the following paper or the github repository.

License

This project is licensed under the 4-clause BSD License.

DiaboloKiat/bodyparts_ros