This package provides a monocular vision-based person tracking and identification framework for person following robots. It first detects people using tf-pose-estimation, and then track them in the robot space with Unscented Kalman filter with the ground plane information. The combination of Convolutional Channel Features and Online Boosting runs on the top of the tracking module to keep tracking the target person with a re-identification capability. The entire system is designed so that it can be run on a Jetson TX2/Xavier, and it can easily be reproduced and reused on a new mobile robot platform.
- dlib
- flann
- tensorflow
- tf-pose-estimation
You can flush your TX2/Xavier with the following image file. It contains all the required packages.
We tested this package on TX2/Jetpack 3.3 and Xavier/Jetpack 4.2.
Put your camera on a fixed place (horizontally) and run the following commands. Change camera_xyz and camera_rpy depending on your camera placement.
roscd monocular_person_following/rviz
rviz -d monocular_person_following.rvizroslaunch monocular_person_following start_robot.launch webcam:=true publish_dummy_frames:=true camera_xyz:="0 0 0.9" camera_rpy:="0 0 0"roslaunch monocular_person_following jetson_person_following.launch camera_name:=/top_front_camera/qhdroscd monocular_person_following/rviz
rviz -d monocular_person_following.rvizroslaunch monocular_person_following start_robot.launch webcam:=false publish_dummy_frames:=true camera_xyz:="0 0 0.9" camera_rpy:="0 0 0"roslaunch monocular_person_following jetson_person_following.launch camera_name:=/csi_cam_0/sd
-
Kenji Koide, Jun Miura, and Emanuele Menegatti, Monocular Person Tracking and Identification with Online Deep Feature Selection for Person Following Robots, Robotics and Autonomous Systems [link].
-
Kenji Koide and Jun Miura, Convolutional Channel Features-based Person Identification for Person Following Robots, 15th International Conference IAS-15, Baden-Baden, Germany, 2018 [link].