TartanCalib contributes state-of-the-art calibration for wide-angle lenses, by implementing an iterative calibration pipeline and adaptive subPixel refinement of AprilTags. The code for TartanCalib is built upon the Kalibr toolbox, is easy to use, and robust. If you are interested in helping us extend the features of TartanCalib, please reach out to Bart (email).
For more information, visit our project page here.
To use the Dockerfiles provided in this repository, ensure that you have Docker installed on your system.
-
Clone and build docker image
git clone https://github.com/castacks/tartancalib cd tartancalib docker build -t tartancalib -f Dockerfile_ros1_20_04 . -
Mount data folder and share xhost within container for GUI. For more information, read the ROS wiki on Docker.
FOLDER=/path/to/your/data/on/host xhost +local:root docker run -it -e "DISPLAY" -e "QT_X11_NO_MITSHM=1" \ -v "/tmp/.X11-unix:/tmp/.X11-unix:rw" \ -v "$FOLDER:/data" tartancalib -
When in the Docker container, follow the Usage section of this readme to run calibration commands.
TartanCalib and Kalibr are integrated within ROS. Ensure that you have ROS installed within your system.
- Install dependencies required for TartanCalib.
Ubuntu 16
sudo apt-get update && apt-get install -y \
git wget autoconf automake \
python2.7-dev python-pip python-scipy python-matplotlib \
ipython python-wxgtk3.0 python-tk python-igraph \
libeigen3-dev libboost-all-dev libsuitesparse-dev \
doxygen \
libopencv-dev \
libpoco-dev libtbb-dev libblas-dev liblapack-dev libv4l-dev \
python-catkin-tools
Ubuntu 18
sudo apt-get update && apt-get install -y \
git wget autoconf automake nano \
python3-dev python-pip python-scipy python-matplotlib \
ipython python-wxgtk4.0 python-tk python-igraph \
libeigen3-dev libboost-all-dev libsuitesparse-dev \
doxygen \
libopencv-dev \
libpoco-dev libtbb-dev libblas-dev liblapack-dev libv4l-dev \
python-catkin-tools
Ubuntu 20
sudo apt-get update && apt-get install -y \
git wget autoconf automake nano \
python3-dev python3-pip python3-scipy python3-matplotlib \
ipython3 python3-wxgtk4.0 python3-tk python3-igraph \
libeigen3-dev libboost-all-dev libsuitesparse-dev \
doxygen \
libopencv-dev \
libpoco-dev libtbb-dev libblas-dev liblapack-dev libv4l-dev \
python3-catkin-tools python3-osrf-pycommon
-
Clone and build repository
mkdir ~/tartan_ws && cd ~/tartan_ws mkdir src && cd src git clone https://github.com/castacks/tartancalib cd .. export ROS1_DISTRO=noetic # kinetic=16.04, melodic=18.04, noetic=20.04 source /opt/ros/$ROS1_DISTRO/setup.bash catkin init catkin config --extend /opt/ros/$ROS1_DISTRO catkin config --merge-devel # Necessary for catkin_tools >= 0.4. catkin config --cmake-args -DCMAKE_BUILD_TYPE=Release catkin build -
Source catkin workspace
cd ~/tartan_ws && source devel/setup.bash -
Follow Usage section to get calibrating!
This can be done through a multitude of different ways. If you have a physical setup linked to ROS, directly recording the sensor stream to a rosbag would be easiest. There are also methods of cutting common video formats into rosbags. The rosbag api is well documented and contains examples that would potentially be helpful if you are new to working with bag files.
Kalibr supports three different calibration targets with different parameters associated to each target. Kalibr's wiki has more information on calibration targets in YAML. An example of an aprilgrid YAML file is provided below:
#example for aprilgrid
target_type: 'aprilgrid' #gridtype
tagCols: 10 #number of apriltags
tagRows: 7 #number of apriltags
tagSize: 0.025 #size of apriltag, edge to edge [m]
tagSpacing: 0.3 #ratio of space between tags to tagSize
#example: tagSize=2m, spacing=0.5m --> tagSpacing=0.25[-]
The method should also work with checkerboards but would require some adaptation. In general grids of AprilTags are most robust.
This is sample code for running calibration with certain parameters changed. Please refer to the table below for the full range of configurable parameters.
In a terminal or bash file:
export ROS1_DISTRO=noetic # kinetic=16.04, melodic=18.04, noetic=20.04
source /opt/ros/$ROS1_DISTRO/setup.bash
In a terminal, for example:
cd ~/tartan_ws && source devel/setup.bash
rosrun kalibr tartan_calibrate \
--bag /path/to/bagfile.bag \
--target /path/to/target.yaml \
--topics /camera_0/image_raw /camera_1/image_raw \
--min-init-corners-autocomplete 29 \
--min-tag-size-autocomplete 2 \
--correction-threshold 10.1 \
--models omni-radtan omni-radtan \
--dont-show-report
--save_dir /path/to/output/
Below is a table of basic parameters to get TartanCalib running. You will minimally have to configure the following parameters.
| Basic Parameter | Description |
|---|---|
| bag | Path to calibration bagfile. |
| target | Path to target YAML files. |
| topics | List of camera topics within bag file to calibrate. If you have three cameras with topic names /camera_n/image_raw, use --topics camera_0/image_raw camera_1/image_raw camera_2/image_raw. Note that ordering here has to match ordering within --models |
| models | Choose the camera/distortion model pair to be fitted. The following choices are currently available: pinhole-radtan, pinhole-equi, pinhole-fov, omni-none, omni-radtan, eucm-none, ds-none. For more details on supported models, check out Kalibr's documentation. |
| log_dest | Save filename of logs. All output filenames will begin with this parameter. (Default: log) |
| save_dir | Save directory of YAML files containing calibration information, and a pickle with log data. Must end with a trailing slash. (Default: /data/) |
| debug_image_dir | Save directory of PNG files for debugging. Must be an existing folder, and debug_image_dir must end with a trailing slash if a folder is specified. Images are saved by default to your workspace folder. (Default: '') |
Below is a table of advanced parameters if you'd like to have greater control over TartanCalib's algorithm.
| Advanced Parameter | Description |
|---|---|
| outputMatlab | When selected, outputs Matlab matrix for use in BabelCalib (or similar). |
| projections | Choose from four possible modes: cornerpredictor, pinhole, homography, and none. Cornerpredictor is the autocomplete method described within the paper, and is recommended for best results. |
| debug-modes | Choose from 7 possible debug modes for additional debug images: pointcloudprojection, pinholeprojection, originalprojection, targetpointcloud, individualprojections, reprojectionpoints |
| fovs | When using pinhole projection mode, this parameter represents FOV of pinhole. This argument accepts multiple pinholes corresponding to each topic. For example, if your bag contains two topics, /camera_0/image_raw, and /camera_1/image_raw, to generate three pinholes of FOVs 30, 60, 90 for camera_0 and one pinhole of fov 90 for camera_1, use --fovs 30 60 90 --fovs 90. Note that the number of arguments for fovs must correspond with poses and resolutions. |
| poses | When using pinhole projection mode, this parameter represents pose of pinhole. As with fovs, this argument can be configured for multiple projections. |
| resolutions | When using pinhole projection mode, this parameter represents resolution of pinhole. As with fovs, this argument can be configured for multiple projections. |
| min-init-corners-autocomplete | The algorithm minimally requires this number of corners for autocomplete. With values that are too small, the pose of the board might be too uncertain for accurate results. (Default: 24) |
| min-tag-size-autocomplete | Minimum size (px) a tag has to be before being autocompleted. This ensures small tags below a certain size are excluded instead of being detected poorly and affecting calibration. (Default: 0.0) |
| correction-threshold | Offset (px) between reprojection and detection allowed. (Default: 10.0) |
| min-resize-window-size | Minimum window size (px) allowed during dynamic sizing. (Default: 2) |
| max-resize-window-size | Maximum window size (px) allowed during dynamic sizing. (Default: 8) |
| symmetry-refinement | If included, symmetry refinement, an experimental feature, will be enabled. |
| symmetry-edge-threshold | When symmetry refinement is active, checks if detections are too close to border of pixels. (Default: 10.0) |
The output YAML files will be found in the directory you've specified in save_dir.
TartanCalib runs in two iterations. The output file using enhancements from TartanCalib may be found in log1-camchain.yaml. The YAML file will contain distortion coefficients for the distortion model used, as well as intrinsics for the camera being calibrated. For multi-camera calibration, the YAML file will also include extrinsics between different cameras.
For more information on the output format, refer to Kalibr's wiki.
The calibration approaches used in TartanCalib are documented in the following paper. Please cite this paper, and the appropriate papers on Kalibr's repository when using this toolbox or parts of it in an academic publication.
@article{duisterhof2022tartancalib,
title={TartanCalib: Iterative Wide-Angle Lens Calibration using Adaptive SubPixel Refinement of AprilTags},
author={Duisterhof, Bardienus P and Hu, Yaoyu and Teng, Si Heng and Kaess, Michael and Scherer, Sebastian},
journal={arXiv preprint arXiv:2210.02511},
year={2022}
}
This work is built upon the Kalibr toolbox.
Copyright (c) 2022 AirLab All rights reserved.
Redistribution and use in source and binary forms, with or without modification, are permitted (subject to the limitations in the disclaimer below) provided that the following conditions are met:
-
Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.
-
Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.
-
Neither the name of the copyright holder nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission.
NO EXPRESS OR IMPLIED LICENSES TO ANY PARTY'S PATENT RIGHTS ARE GRANTED BY THIS LICENSE. THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
Copyright (c) 2014, Paul Furgale, Jérôme Maye and Jörn Rehder, Autonomous Systems Lab, ETH Zurich, Switzerland
Copyright (c) 2014, Thomas Schneider, Skybotix AG, Switzerland
All rights reserved.
Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:
-
Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.
-
Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.
-
All advertising materials mentioning features or use of this software must display the following acknowledgement: This product includes software developed by the Autonomous Systems Lab and Skybotix AG.
-
Neither the name of the Autonomous Systems Lab and Skybotix AG nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTONOMOUS SYSTEMS LAB, SKYBOTIX AG ''AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE AUTONOMOUS SYSTEMS LAB OR SKYBOTIX AG BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.