Object Analytics (OA) is ROS2 wrapper for realtime object detection, localization and tracking. These packages aim to provide real-time object analyses over RGB-D camera inputs, enabling ROS developer to easily create amazing robotics advanced features, like intelligent collision avoidance and semantic SLAM. It consumes sensor_msgs::PointClould2 data delivered by RGB-D camera, publishing topics on object detection, object tracking, and object localization in 3D camera coordination system.
OA keeps integrating with various "state-of-the-art" algorithms.
- Object detection offload to VPU, ros_intel_movidius_ncs, with MobileNet SSD model and Caffe framework
ROS2 packages from ROS2
- ament_cmake
- std_msgs
- sensor_msgs
- geometry_msgs
- rclcpp
- rosidl_default_generators
- rosidl_interface_packages
- launch
- ros2run
- class_loader
- cv_bridge
- object_msgs
- ros2_intel_movidius_ncs
- pcl_conversions
- ros2_message_filters
Other non-ROS packages
- libpcl-all
- libpcl-all-dev
- python3-numpy
- OpenCV3 (Minimum reqired OpenCV3.2, Kinetic comes with 3.3)
cd ${ros_ws} # "ros_ws" is the ament workspace root directory where this project is placed in
ament build --symlink-install --isolated --only-packages object_analytics_node object_analytics_msgs object_analytics_launchYou might don't need "--symlink-install" and "--isolated" opitions, it totally depends on your own build style.
The only supported RGB-D camera by now is Intel RealSense
realsense_ros2_camera
Movidius NCS is the only supported detection backend
echo -e "param_file: mobilenetssd.yaml\ninput_topic: /object_analytics/rgb" > `ros2 pkg prefix movidius_ncs_launch`/share/movidius_ncs_launch/config/default.yaml
launch `ros2 pkg prefix object_analytics_launch`/share/object_analytics_launch/launch/analytics_movidius_ncs.py/object_analytics/rgb (sensor_msgs::msg::Image)
/object_analytics/pointcloud (sensor_msgs::msg::PointCloud2)
/object_analytics/localization (object_analytics_msgs::msg::ObjectsInBoxes3D)
/object_analytics/tracking (object_analytics_msgs::msg::TrackedObjects)
/movidius_ncs_stream/detected_objects (object_msgs::msg::ObjectsInBoxes)
As currently RViz2 is not ready for object analytics visualization, we use view result in RViz on the ROS1 side via ros1_bridge. Steps to be able visualize object analtyics result on RViz are as following.
-
Rebuild ros1_bridge following official guide with ROS2 object_analytics_msgs has already built.
-
Patch and rebuild ROS1's object_analytics_visualization due to name changes.
#in your ros1 workspace which has object_analytics_visualization source code
cd <path/to/object_analytics/root>
git apply ros1_bridge-patch.patch
# you can find path following [link](https://github.com/intel/ros2_object_analytics/blob/master/patch/ros1_bridge-patch.patch)
catkin_make --only-pkg-with-deps object_analytics_visualization
catkin_make install- Run
# Shell A:
. /opt/ros/kinetic/setup.bash
roscore# Shell B:
. /opt/ros/kinetic/setup.bash
. <install-space-with-bridge>/local_setup.bash
export ROS_MASTER_URI=http://localhost:11311
ros2 run ros1_bridge dynamic_bridge# Shell C
. /opt/ros/kinetic/setup.bash
. <install-space-with-object-analytics-visualization>/setup.bash
roslaunch object_analytics_visualization rviz.launch# Shell D
. <install-space-with-realsense-ros2-camera>/local_setup.bash
realsense_ros2_camera
# Shell E
. <install-space-with-object-analytics-launch>/local_setup.bash
echo -e "param_file: mobilenetssd.yaml\ninput_topic: /object_analytics/rgb" > `ros2 pkg prefix movidius_ncs_launch`/share/movidius_ncs_launch/config/default.yaml
launch `ros2 pkg prefix object_analytics_launch`/share/object_analytics_launch/launch/analytics_movidius_ncs.py
By default, object analytics will launch both tracking and localization features, but either tracking or localization or both can be dropped. Detailed please refer comments embedded in launch file.