Installation

The VIO is implemented in ROS. Thus, you first need to install ROS Indigo according to the official instructions.

You will also need PySide for Python 2.7

sudo apt-get install python-pyside

For the visualization, youalso need PyQtGraph. Download the .deb file from the website and double click it to install.

The VIO depends on the following ROS packages

This package for the state estimation,
ait_ros_messages for custom ROS messages,
klt_feaute_tracker for feature tracking,
A ROS driver that publishes image and IMU data in VioSensorMsg messages as defined in ait_ros_messages. Depending on the camera you are using, this package will differ.
- If you are using a DUO3d camera, the package you need is duo3d_ros. You will also need to place the DUO SKD in catkin_ws/src, which customers can download from the DUO3d website

Clone all git repositories into your catkin_ws/src directory.

You are now ready to compile the packages with

cd catkin_ws
catkin_make

Test Data

We recorded two test ROS bags with the Duo Callibration file. The data can be downloaded here.

Compiling for embedded devices

A version of the algorithm optimized for ARM Cortex-A devices is provided. To build this optimized version, open the CMakeLists.txt of this package and uncomment the line:

set(CORTEX 1)

You will need to have NEON 10 installed. Follow the installation instructions provided on the website and modify the NEON_PATH variable in CMakeLists.txt to point to the appropriate location.

Calibration

Note: It is recommended that you first start a roscore that is always running. This makes it easier for ROS nodes to communicate with each other if some of them have to be restarted.

roscore

Camera Calibration

Calibrate the stereo camera using the DUOCalibrator in the duo3d_ros package. First start the camera with:

roslaunch duo3d_ros duo.launch

Then, run the calibrator:

rosrun duo3d_ros DUOCalibrator.py

Choose the appropriate checkerboard size and dimensions. You can figure out the checkerboard size by pointing the camera at the checkerboard and varying the rows and columns in the app until the whole checkerboard is detected and rainbow colored.

Once the settings are correct, calibrate the camera by pressing the Start recording button and capturing the checkerboard form various angles and distances. It is recommended that you take at least 30 images. Once you feel that the checkerboard has been recorded from enough poses and appears in all areas of the images, press Stop recording. The cameras will be calibrated (this can take several seconds).

IMU Calibration

Once you have calibrated the camera, you need to calibrate the IMU. For this, place the camera on a level surface, such that the camera has zero roll and pitch: The camera should look horizontally forward.

It it's not already running, start the camera with:

roslaunch duo3d_ros duo.launch

Run the calibrator with:

rosrun duo3d_ros IMUCalibrator.py

Follow the command line instructions to calibrate the camera and write the calibration to the appropriate calibration file.

Using the VIO

Once both the camera and the IMU has been calibrated, we are ready to use the system. Start the visualization (if it isn't already):

roslaunch duo_vio vis.launch

Run the VIO:

roslauch duo_vio duo_vio.launch

Camera Lenses

The standard lenses provided with the DUO3d stereo camera have a wide field of view which is not ideally suited for VIO applications. It is therefore recommended that you replace them with ones with a narrower field of view. We use FS3028B13M8 lenses by Focusafe wich can be purchased on alibaba.com.

License

This package contains code generated by the Matlab Coder Toolbox. The code is generated with a Matlab academic license. As a consequence, the generated code is suitable for academic purposes only and may not be used for commercial purposes. The license terms for this code are as follows:

Academic License - for use in teaching, academic research, and meeting course requirements at degree granting institutions only. Not for government, commercial, or other organizational use.

All code not generated by Matlab Coder is subject to the BSD license.

ethz-ait/duo_vio