/Surround-View

Primary LanguageC++MIT LicenseMIT

dependencies

  1. apt install nvidia-cuda-toolkit nvidia-cuda-toolkit-gcc
  2. apt install libglm-dev libglfw3-dev libassimp-dev libegl-dev libgles-dev
  3. opencv 4.x with cuda support
  4. if using assimp-5.2, copy the config file to let cmake find it: cp /usr/lib/x86_64-linux-gnu/cmake/assimp-5.2/assimpConfig.cmake /usr/lib/x86_64-linux-gnu/cmake/assimp-5.2/assimp-config.cmake

The 3D surround view on Jetson Tegra TX2 platform - this is my research project for master thesis.

The essence of the project is to build a good quality stitching of images from 4 cameras and project the image onto a mesh model with acceptable performance. The project work on ~20 fps with constraints from the connection adaptor 30 fps due the synchronize frames mode.

Cite(related paper):

@incollection{gasparyan2022real,
  title={Real-Time 3D Surround View System for Vehicle Based on Panoramic Stitching Image},
  author={Gasparyan, Sokrat and Vasilianov, Georgii},
  booktitle={Networked Control Systems for Connected and Automated Vehicles: Volume 1},
  pages={85--93},
  year={2022},
  publisher={Springer}
}

The 3D bowl view:

The top view:

The hardware setup:

Device type Device name
Camera e-CAM30A CUMI0330 MOD
Connection adaptor board e-CAM130 TRICUTX2 ADAPTOR
Connecting wires High-speed 2-lane MIPI CSI-2
Embedded platform Jetson Tegra TX2
Tripod -

The software setup:

  • OS - Linux Ubuntu LTS v16.04
  • C++ 14, Python 3.6, GLSL, C CUDA
  • CMake >= v3.16
  • CUDA Toolkit v9.0
  • V4L2 driver
  • OpenGL ES v3.2
  • OpenMP

The data processing flowchart

The 3d party library:

3D Car model