JazzyFeng/LAIS

A learning-based autonomous inspection system

LAIS

Learning-based Autonomous Inspection UAV System

LAIS is a learning-based automonous inspection system developed for quadrotor platform. In this work, it consists of 2-D object detector, 3-D object state estimation and path planning. The 2-D object detector support YOLO series. The 3-D object state estimation and flight trajectory can be visualized by Rviz in real-time.

Video

Publications

Feng, Y., Tse, K., Chen, S., Wen, C. Y., & Li, B. (2021). Learning-Based Autonomous UAV System for Electrical and Mechanical (E&M) Device Inspection. Sensors, 21(4), 1385.

Requirements

• Ubuntu 16.04 or 18.04
• ROS Kinetic or Melodic: ROS Install
• OpenCV >= 4.4: OpenCV Linux Install
• Python 3.8
• CUDA >= 10.0: CUDA Toolkit Archive
• CUDNN >= 7.0: cuDNN Archive

Build instructions on Linux

1. clone repository into working space

cd ~/catkin_ws/src
git clone https://github.com/JazzyFeng/LAIS.git

2. Install 3rd Party library

cd ~/catkin_ws/src/LAIS/3rdPartLib/
./install3rdPartLib.sh

3. Compile

cd ~/catkin_ws
catkin_make

How to use (to detect objects in our experiment)

1. Download Yolov4.cfg, Yolov4.weight and obj.names
2. Change these lines :

• cfg_path
• weight_path
• classid_path

static string cfg_path
static string weight_path
static string classid_path

3. Compile and launch camera node:

cd ~/catkin_ws
catkin_make
rosrun LAIS camera

4. To visualize 2-D bounding boxes, uncomment the line drawBoundingBox

yolo.drawBoundingBox(image_rgb);

5. To visualize the estimate of object position in inertial frame, launch rviz node:

cd ~/catkin_ws
roslaunch rviz.launch

6. To improve the detection speed or accuracy, change the default input size yoloNet

static yoloNet yolo = yoloNet(cfg_path, weight_path, classid_path, 608, 608, 0.5);

How to use (to inspect objects in our experiment)

1. Connect the quadrotor with flight controller and launch mavros

roslaunch mavros px4.launch 

2. Launch the D435i camera by realsense-ros

roslaunch realsense2_camera rs_camera.launch 

3. Launch the camera node and fj005 node:

cd ~/catkin_ws
roslaunch LAIS fj005.launch

4. To visualize the detected object and flight trajectory in inertial frame, launch rviz node:

cd ~/catkin_ws
roslaunch rviz.launch

Maintainer

Yurong Feng(Dept.ME,PolyU): yurong.feng@connect.polyu.hk