This is a ROS package developed for object detection in camera images. Using darknet_ros(YOLO), for real-time object detection system. And using jsk_pcl for estimation coordinates of detected objects, just one class. NOT SUPPORT MULTI CLASSES. In the following ROS package detecting "plier", "hammer", "screw_driver". for this, I've ref. the open-manipulator-object-tracking-apple.
The packages have been tested under ROS Kinetic and Ubuntu 16.04, OpenCV 3.4.2, NVIDIA-SMI 418.87.00 Driver Version: 418.87.00 CUDA Version: 10.1
Author: Hyeonjun Park, koreaphj91@gmail.com
Affiliation: Human-Robot Interaction LAB, Kyung Hee Unviersity, South Korea
- Before do this, please backup important files.
This software is built on the Robotic Operating System (ROS).
One line install: https://cafe.naver.com/openrt/14575
for Desktop
wget https://raw.githubusercontent.com/ROBOTIS-GIT/robotis_tools/master/install_ros_kinetic.sh && chmod 755 ./install_ros_kinetic.sh && bash ./install_ros_kinetic.sh
Additionally, YOLO for ROS depends on following software:
- OpenCV 3.4.x (computer vision library),
- OpenCV 3.4.2 (How to install the opencv? in Korean),
$ sudo apt-get update && sudo apt-get upgrade
-- installation the env.
$ sudo apt-get -y purge libopencv* python-opencv
$ sudo apt-get -y install build-essential cmake vim
$ sudo apt-get -y install pkg-config libjpeg-dev libtiff5-dev libjasper-dev libpng12-dev libavcodec-dev
$ sudo apt-get -y install libavformat-dev libswscale-dev libxvidcore-dev libx264-dev libxine2-dev libv4l-dev
$ sudo apt-get -y install v4l-utils libgstreamer1.0-dev libgstreamer-plugins-base1.0-dev libqt4-dev
$ sudo apt-get -y install libgtk2.0-dev libgtk-3-dev mesa-utils libgl1-mesa-dri libqt4-opengl-dev
$ sudo apt-get -y install libatlas-base-dev gfortran libeigen3-dev python3-dev python3-numpy python-dev python-numpy libatlas-base-dev gfortran
-- opencv 3.4.2 & opencv_contrib-3.4.2 download
$ wget -O opencv.zip https://github.com/opencv/opencv/archive/3.4.2.zip
$ unzip opencv.zip
$ wget -O opencv_contrib.zip https://github.com/opencv/opencv_contrib/archive/3.4.2.zip
$ unzip opencv_contrib.zip
-- opencv build
$ cd ~/opencv-3.4.2/
$ mkdir build
$ cd build
$ cmake -D CMAKE_BUILD_TYPE=RELEASE \
-D CMAKE_INSTALL_PREFIX=/usr/local \
-D WITH_TBB=OFF \
-D WITH_IPP=OFF \
-D WITH_1394=OFF \
-D BUILD_WITH_DEBUG_INFO=OFF \
-D BUILD_DOCS=OFF \
-D INSTALL_C_EXAMPLES=ON \
-D INSTALL_PYTHON_EXAMPLES=ON \
-D BUILD_EXAMPLES=OFF \
-D BUILD_TESTS=OFF \
-D BUILD_PERF_TESTS=OFF \
-D WITH_QT=OFF \
-D WITH_GTK=ON \
-D WITH_OPENGL=ON \
-D OPENCV_EXTRA_MODULES_PATH=../../opencv_contrib-3.4.2/modules \
-D WITH_V4L=ON \
-D WITH_FFMPEG=ON \
-D WITH_XINE=ON \
-D BUILD_NEW_PYTHON_SUPPORT=ON \
../
-- opencv make compile
$ time make
-- opencv install
$ sudo make install
- CUDA 10.0 with cuDNN 10.x, (How to install the cuda? in Korean),
- notice: Do it after install the nvidia driver. And the cuda version is follow on your GPU. check up the GPUs supported
- Download the CUDA in the NVIDIA webpage (link)
$ sudo dpkg -i cuda-repo-ubuntu1604-10-0-local-10.0.130-410.48_1.0-1_amd64.deb
$ sudo apt-key add /var/cuda-repo-<version>/7fa2af80.pub
$ sudo apt-get update
$ sudo apt-get install cuda
$ gedit ~/.bashrc
export PATH=/usr/local/cuda-10.0/bin${PATH:+:${PATH}}$
export LD_LIBRARY_PATH=/usr/local/cuda-10.0/lib64${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}
Or, if you installed the cuda 10.1,
$ gedit ~/.bashrc
export PATH=/usr/local/cuda-10.1/bin${PATH:+:${PATH}}$
export LD_LIBRARY_PATH=/usr/local/cuda-10.1/lib64${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}
$ reboot
- check up the installed
$ cat /proc/driver/nvidia/version
NVRM version: NVIDIA UNIX x86_64 Kernel Module 418.87.00 Thu Aug 8 15:35:46 CDT 2019
GCC version: gcc version 5.4.0 20160609 (Ubuntu 5.4.0-6ubuntu1~16.04.11)
$ nvcc -V
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2018 NVIDIA Corporation
Built on Sat_Aug_25_21:08:01_CDT_2018
Cuda compilation tools, release 10.0, V10.0.130
- After download the cuDNN
$ sudo dpkg -i libcudnn7_7.6.2.24-1+cuda10.0_amd64.deb
$ sudo dpkg -i libcudnn7-dev_7.6.2.24-1+cuda10.0_amd64.deb
$ sudo dpkg -i libcudnn7-doc_7.6.2.24-1+cuda10.0_amd64.deb
-
darknet YOLO, (How to install the YOLO? in Korean),
-
darknet_ros (How to install the darknet_ros? in Japanese)
$ cd ~/catkin_ws/src
$ git clone --recursive https://github.com/leggedrobotics/darknet_ros.git
$ cd ..
$ catkin_make darknet_ros
$ cd ~/catkin_ws/devel
$ source setup.bash
Intel SDK 2.0 install
- Register the server's public key
$ sudo apt-key adv --keyserver keys.gnupg.net --recv-key F6E65AC044F831AC80A06380C8B3A55A6F3EFCDE || sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-key F6E65AC044F831AC80A06380C8B3A55A6F3EFCDE
- Add the server to the list of repositories:
$ sudo add-apt-repository "deb http://realsense-hw-public.s3.amazonaws.com/Debian/apt-repo xenial main" -u
$ sudo apt-get install librealsense2-dkms
$ sudo apt-get install librealsense2-utils
$ sudo apt-get install librealsense2-dev
$ sudo apt-get install librealsense2-dbg
RealSense ROS
$ cd ~/catkin_ws/src/realsense
$ git clone https://github.com/pal-robotics/ddynamic_reconfigure.git
sudo apt-get install ros-kinetic-jsk-recognition
sudo apt-get install ros-kinetic-jsk-topic-tools
source biuld pacakges
$ cd ~/catkin_ws/src
$ git clone https://github.com/jsk-ros-pkg/jsk_common.git
$ cd jsk_common
$ rosdep install -y -r --from-path . --ignore-src
$ cd ..
$ catkin_make darknet_ros
- nodelet, rtabmap-ros, navigation
$ sudo apt-get install ros-kinetic-octomap-server
$ sudo apt-get install ros-kinetic-nodelet
$ sudo apt-get install ros-kinetic-depth-image-proc
$ sudo apt-get install ros-kinetic-rtabmap-ros
$ sudo apt-get install ros-kinetic-navigation
ref. page, Oroca naver cafe, Korean
Copy and paste
$ roscd hj_object_detect/
$ cd copy_paste_files
$ cp -r hj_p.yaml ~/catkin_ws/src/darknet_ros/darknet_ros/config
$ cp -r p.cfg ~/catkin_ws/src/darknet_ros/darknet_ros/yolo_network_config/cfg
$ cp -r YoloObjectDetector.cpp ~/catkin_ws/src/darknet_ros/darknet_ros/src
$ cp -r YoloObjectDetector.hpp ~/catkin_ws/src/darknet_ros/darknet_ros/include/darknet_ros
Weight file Dowload link
$ cd (your dowload folder)
$ cp -r p_final.weights ~/catkin_ws/src/darknet_ros/darknet_ros/yolo_network_config/weights
Real realsense camera
$ roslaunch hj_object_detect hj_object_detect_rviz.launch
$ roslaunch hj_object_detect hj_jsk_test.launch
Gazebo simulation
$ roslaunch hj_object_detect hj_object_detect_rviz.launch sim:=true
$ roslaunch hj_object_detect hj_jsk_test.launch sim:=true
- bringup_d435.launch
- rs_rgbd.launch
- hj_darknet.launch
- config: $(find darknet_ros)/yolo_network_config/weights
- weight: $(find darknet_ros)/yolo_network_config/cfg
- parameter: $(find hj_object_detect)/config/hj_darknet_config.yaml
- parameter: $(find darknet_ros)/config/hj_p.yaml
- point cloud nodelet
- jsk_pcl_utils/LabelToClusterPointIndices nodelet
- jsk_pcl/ClusterPointIndicesDecomposer
- If the coordinate of detected hammer object is not visible, try to this process.
$ cd ~/catkin_ws/src
$ cd jsk_common
$ rosdep install -y -r --from-path . --ignore-src
Click image to link to YouTube video.
