/UE_SensorSimulator

Unreal Engine Plugin for Synthetic Data Generation

Primary LanguagePython

UE_SensorSimulator

Unreal Engine Plugin for Synthetic Data Generation

Features

  • Realtime lidar sensor (scanning the world level via LineTracer)
  • Depth map, Point cloud (optional color mapping for attached camera), Camera RGB image
  • Supporting async generation
  • Supporting UDP communication for the sensor data

Quick Install & Setup

  1. Download Latest Release
  2. Create new or choose project.
  3. Browse to your project folder (typically found at Documents/Unreal Project/{Your Project Root})
  4. Copy Plugins folder into your Project root.
  5. Plugin should be now ready to use.

DEMO

Overview

Image 1

Digital Twin

Implementing corner case scenarios such as more than 40 weather, brightness changes according to time, and extreme weather

Image 1 Image 2
Image 3 Image 4

Segmetation

Collecting synthetic data for training the Segmentation model (PIDNet) and utilizing Unreal Engine's stencil buffer functionality to extract Semantic maps, which will be used as target data.

Image 1

Point Cloud

Implementing an LIDAR sensor in Unreal Engine to capture RGB and Depth information, transmitting this data to Python via UDP communication, and then implementing a Point Cloud in Panda3D.

Image 1

Sensor Calibration

Calibrating the PointCloud generated by the Lidar Sensor with the imagery from the RGB Sensor to create a reference image. Using this calibrated image to enhance the reliability and accuracy of the depth estimation model.

Image 1