/ROS-based-3D-detection-Tracking

Deployment of 3D-Detection and Tracking pipeline in simulation based on rosbags and real-time.

Primary LanguagePythonApache License 2.0Apache-2.0

ROS-based-3D-detection-Tracking

Implementation results on rosbags

Image 1 Description Image 2 Description

Scenario 1: Detecting multiple objects in the scene Scenario 2: Accurately retaining detection and track ids

Image 3 Description
Scenario 3: Detecting objects in the reverse direction

Image 4 Description
Scenario 4: Detecting a virtual vehicle on point-cloud data

Deployment results

Image 6 Description Image 7 Description

Scene: Pointpillar + Regnet Scene: Pointpillar (TensorRT)

Backbone mAP (nuScenes) FPS Memory (GPU)
Pointpillars 39.26 8.88 3.5 Gb
Pointpillars (TensorRT) ---- 15.15 4.5 Gb
Pointpillar + Regnet 48.2 7.76 3.5 Gb

Installation

  1. Create virtual environment
conda create --name rosdeploy python=3.8
conda activate rosdeploy
  1. Install pytorch conda install pytorch from https://pytorch.org/ according to cuda version or lower for eg.
conda install pytorch==1.8.0 torchvision==0.9.0 torchaudio==0.8.0 cudatoolkit=11.1 -c pytorch -c conda-forge
  1. Clone this repository
git clone git@github.com:ksm26/ROS-based-3D-detection-Tracking.git
  1. Install steps
pip install -U openmim
mim install mmcv-full
mim install mmdet
mim install mmsegmentation
  1. Install mmdetection3d
cd mmdetection3d
pip install -e .
  1. Install mmdeploy
cd mmdeploy 
pip install -e .
  1. Install tf and jsk tf tutorial Bounding box visualization ROS jsk_visualization jsk_recognition_msgs
sudo apt-get install ros-melodic-jsk-recognition-msgs 
sudo apt-get install ros-melodic-jsk-rviz-plugins
sudo apt-get install ros-melodic-jsk-rqt-plugins
sudo apt-get install ros-melodic-jsk-visualization
  1. Install tensorrt
pip install nvidia-tensorrt
python3 -m pip install --upgrade tensorrt

Additional installation

Error with rospy.init_node(): An error with ros packages installation into conda environment. Run the above ros_python_environment.yml using:

pip install scikit-build
conda env update --file ros_python_environment.yml

import tf: Error
Import tf does not work with python 3 as it is compiled for python 2. Solution: Recompile with python3 in catkin_workspace
To be able to work with python interpreter (pycharm,visual studio) add the compile path as below:

sys.path.append("~/Desktop/ROS-tracker/catkin_ws/src:/opt/ros/melodic/share")

Source catkin envrionment after tf installation and creation of new catkin workspace

source ~/catkin_ws/devel/setup.bash

Benchmark and Model Zoo

Download the checkpoint for 3D-detection model-zoo. This repository evaluates the performance of Pointpillars and Regnet models pretrained on nuScences dataset.
Test by running a rosbag and subscribe to appropriate rostopics

rosbag play rosbag_name.bag
python3 mmdetection3d/demo/pcd_demo_class.py

Convert the checkpoint into ONNX file and than to tensorRT .engine file

python3 mmdeploy/tools/deploy.py

ONNX file can be visualized using NETRON.

Licence

This project is released under the Apache 2.0 license.

Reference