We introduce burst feature finder (BuFF), a 2D + time feature detector and descriptor that finds features with well defined scale and apparent motion within a burst of frames. We reformulate the trajectory of a robot as multiple burst sequences and perform 3D reconstruction in low light.
- BuFF: Burst Feature Finder for Light-Constrained 3D Reconstruction
- submitted for oral presentation at ICRA 2023
- Authors: Ahalya Ravendran, Mitch Bryson, and Donald G Dansereau
- website: roboticimaging.org/BuFF with dataset details, digestable results and visualizations
Note: The code and visualisations of our ICRA2021 paper "Burst imaging for light-constrained structure-from-motion" (Burst with Merge) used in this paper for comparison is available at roboticimaging.org/BurstSfM
BuFF is built with MATLAB and tested on >= R2021a versions. This repository includes code for both variations of BuFF feature extraction. We extend SIFT feature extraction to find features in a higher dimensional search space. Our implementation is inspired by SIFT implementation by VLFeat Library and LiFF implementation. The functional dependencies required for evaluation and more details are discussed in 'requirements.txt':
git clone https://github.com/RoboticImaging/BuFF/
cd BuFF/The toolkit consists of the following sub-modules.
- assets: Contains the source files required for creating the repository.
- common: Common scripts to run both variants of feature extraction on standard datasets.
- main: Scripts to run two variants of the feature extractor: Burst with 1D apparent motion and Burst with 2D apparent motion.
- utils: General utility functions for e.g. burst visualisation, histogram equalization.
We evaluate our feature extractor on a dataset collected in light-constrained environment using UR5e robotic arm. To download the complete dataset and an example seperately refer to the following links:
| Images | Dataset |
|---|---|
| Dataset description | Read me |
| Example burst | a burst of noisy images and corresponding ground truth with 1D and 2D apparent motion here (2.1GB) |
| Dataset with 1D apparent motion |
dataset including ground truth and noisy images here (40.3GB) |
| Dataset with 2D apparent motion |
dataset including ground truth and noisy images here (40.3GB) |
Preparation: Download the dataset from above and unpack the zip folder. Select the directory in which images are stored and perform bias correction for accurate results.
We have now added python support for BuFF implementation in the sub-modules python
conda create -n buffenv
conda activate buffenv
pip install opencv-python numpy
cd BuFF/python/
python3 BuFF.pyPlease consider citing our paper if you use any of the ideas presented in the paper or code from this repository:
@inproceedings{ravendran2022burst,
author = {Ahalya Ravendran and
Mitch Bryson and
Donald G Dansereau},
title = {{BuFF: Burst Feature Finder for Light-Constrained 3D Reconstruction}},
booktitle = {arXiv},
year = {2022},
}
We use some functions directly from LFToolbox for visualisation.