Simultaneous Synchronization and Calibration for Wide-baseline Stereo Event Cameras

ICRA2024 Paper Submission

This repository hosts the codebase and dataset for our research paper, "Simultaneous Synchronization and Calibration for Wide-baseline Stereo Event Cameras," currently under review for ICRA2024.

Abstract

Event-based cameras offer remarkable advantages such as high temporal resolution and low power consumption but suffer from synchronization issues when deployed in multi-camera settings. Our paper introduces a software-based method to achieve millisecond-level synchronization while simultaneously estimating extrinsic parameters. Our approach eliminates the need for specialized hardware, thus making it particularly suitable for wide-baseline configurations. The robustness and applicability of our method are empirically demonstrated through extensive simulations and real-world experiments.

Illustration of dual-perspective event capture

Temporal misalignment

Usage

To get started, follow the step-by-step instructions below:

Dataset Download:
- Begin by downloading our datasets from our provided OneDrive link.
- Additionally, if you wish to simulate more data yourself, you can leverage the configurations we provide and use the ESIM tool, available at ESIM's GitHub Repository.
Environment Setup:
- Navigate to the ./config directory where you can find the ssac.yaml file.
- Use this yaml file to create a conda environment by running:
```
conda env create -f ssac.yaml
```
- Activate the newly created environment:
```
conda activate ssac
```
MATLAB Engine API Setup:
- Locate your MATLAB root directory. Once located, navigate to the external engines python directory by running:
```
cd "matlabroot/extern/engines/python"
```
- Inside this directory, set up the MATLAB Engine API by executing:
```
python setup.py install
```
Running the Code:
- Navigate to the ./src directory.
- To execute the main script, run:
```
python esim_ssac.py
```
- The configuration settings within the script can be easily modified to suit your needs or to experiment with different parameters.

Results

The robustness of our proposed synchronization approach is empirically substantiated through extensive evaluations, including both simulation-based benchmarks and real-world experimental validations.

Unsynchronized event streams and F matrix

Synchronized event streams and F matrix

Contributions

Novel software-based approach for temporal synchronization of event-based cameras in wide-baseline settings.
Simultaneous estimation of extrinsic parameters, thus integrating these processes for increased efficiency.
Comprehensive validation of the method through simulations and real-world experiments.

Contact

For further inquiries or questions, please contact us at [wlxing@connect.hku.hk].

Thank you for your interest in our research.