/hdr_bracketing_cam_ctrl

[ICRA'24] An Image Acquisition Scheme for Visual Odometry based on Image Bracketing and Online Attribute Control

Primary LanguageC++

HDR Bracketing Camera Attribute Control

This is an official repository of

An Image Acquisition Scheme for Visual Odometry based on Image Bracketing and Online Attribute Control, Shuyang Zhang, Jinhao He, Bohuan Xue, Jin Wu, Pengyu Yin, Jianhao Jiao and Ming Liu.

This paper will be officially released on IEEE International Conference on Robotics and Automation (ICRA) 2024.

System Overview

System Overview

Highlights

  • A camera attribute control method adapted to image bracketing patterns. Images with various exposures are captured for scene exploration, and optimal exposure for the next control is globally optimized by Gaussian process regression (GPR).
  • A VO-oriented image acquisition scheme that explores a wide dynamic range and provides stable image sequences in the time domain. The system leverages the exploration of the dynamic range with system output frequency according to the bracketing pattern design.

image bracketing pattern

Image Bracketing Pattern (Bracket of 4)

Basic Information

  • This work focus on the camera attribute control task. The attributes that we focus on include the exposure time and the analog gain.
  • Since this work needs an interaction with the camera device (to control the camera's attributes), we need a camera entity with image bracketing interface. We use the FLIR BFS cameras (FLIR BFS-U3-31S4C) and the bracketing images are captured by the Sequencer within the Spinnaker API.
  • We implement our method alongside several baseline methods. For real-time usage, we package them as a ROS node which can be considered as a camera driver.
    • If you have a suitable FLIR BFS camera, you can follow the Getting Start to run this driver directly on your FLIR BFS devices.
    • If you have a suitable camera device with image bracketing interface but not FLIR with Spinnaker API, you can follow the guide to extract our method from the code and reimplement it on your own platforms.

Requirements

Hardware

Software

Getting Start

  1. Install ROS (Noetic Ninjemys)
  2. Install Spinnaker
  3. Generate ROS workspace
mkdir ${Workspace_PATH}\src
  1. Pull this repository with 3rdparty modules
cd ${Workspace_PATH}\src
# use '--recursive' to add 3rdparty modules
git clone git@github.com:ShuyangUni/hdr_bracketing_cam_ctrl.git --recursive
  1. Compile this repository
cd ${Workspace_PATH}
catkin_make
  1. Set YAML parameters following this guide
  2. Calibrate the Camera Response Function (CRF), please follow this guide
  3. Run the ROS node
source devel/setup.bash
# change the launch file to your config file
roslaunch hdr_attr_ctrl test_camera_auto.launch

Experiment Results

Platform

platform

Our Platform (4 FLIR BFS-U3-31S4C cameras and 1 Intel NUC11TNKi7)

Visual Odometry

floorplan vo_trajectory

vo_sequences

expo_val freq no_orb_detect no_orb_matched

Drastic Luminance Change

drastic_luminance_change

static_illumination

Citation

If you find this project useful, please cite our paper

@inproceedings{zhang2024image,
  title={An image acquisition scheme for visual odometry based on image bracketing and online attribute control},
  author={Zhang, Shuyang and He, Jinhao and Xue, Bohuan and Wu, Jin and Yin, Pengyu and Jiao, Jianhao and Liu, Ming},
  booktitle={2024 IEEE International conference on robotics and automation (ICRA)},
  year={2024},
  organization={IEEE}
}