Problems with calibrating intrinsic parameters on smartphone camera. Update: issue was wrong charuco board config, 4x3 instead of 3x4
DawidStankiewicz opened this issue · 4 comments
DawidStankiewicz commented
System information (version)
- Operating System / Platform => docker frameau/opencv-ceres, Ubuntu 20.04.2 LTS, (update, on
bailool/mc-calib-prodthe same issues) - OpenCV => 4.2.0
- Ceres => ❔
- Boost => ❔
- C++ => ❔
- Compiler => ❔
Vision system
- Number of cameras => 1 - Samsung Galaxy Note 20 Ultra
- Types of cameras => perspective
- Multicamera configurations => No
- Configuration file =>
######################################## Boards Parameters ###################################################
number_x_square: 4 # number of squares in the X direction
number_y_square: 3 # number of squares the Y direction
resolution_x: 3840 # horizontal resolution in pixel
resolution_y: 2160 # vertical resolution in pixel
length_square: 0.04 # parameters on the marker (can be kept as it is)
length_marker: 0.03 # parameters on the marker (can be kept as it is)
number_board: 1 # number of boards used for calibration (for overlapping camera 1 is enough ...)
boards_index: [ ] # leave it empty [] if the board index are ranging from zero to number_board; example of usage boards_index: [5,10 <-- only two board with index 5/10
square_size: 6.5 # size of each square of the board in cm/mm/whatever you want
############# Boards Parameters for different board size (leave empty if all boards have the same size) #################
number_x_square_per_board: [ ]
number_y_square_per_board: [ ]
square_size_per_board: [ ]
######################################## Camera Parameters ###################################################
distortion_model: 0 # 0:Brown (perspective) // 1: Kannala (fisheye)
distortion_per_camera: [ ] # specify the model per camera, #leave "distortion_per_camera" empty [] if they all follow the same model (make sure that the vector is as long as cameras nb)
number_camera: 1 # number of cameras in the rig to calibrate
refine_corner: 1 # activate or deactivate the corner refinement
min_perc_pts: 0.8 # min percentage of points visible to assume a good detection
cam_params_path: None # file with cameras intrinsics to initialize the intrinsic, write "None" if no initialization available
fix_intrinsic: 0 #if 1 then the intrinsic parameters will not be estimated nor refined (initial value needed)
######################################## Images Parameters ###################################################
cam_prefix: "Cam_"
######################################## Optimization Parameters #############################################
ransac_threshold: 10 # RANSAC threshold in pixel (keep it high just to remove strong outliers)
number_iterations: 1000 # Max number of iterations for the non linear refinement
######################################## Hand-eye method #############################################
he_approach: 0 #0: bootstrapped he technique, 1: traditional he
######################################## Output Parameters ###################################################
save_detection: 1
save_reprojection: 1
camera_params_file_name: "cameras.yml" # "name.yml"- Image sequences => pass
QSp34wGmuD
https://nextcloud.sportvision.tech/s/KSxPWpji9BF8ytj - there are source images and example output.
Describe the issue / bug
I'm trying to calibrate intrinsic parameters of the smartphone camera using MC-Calib. App is often crashing and when it's not then results and logs from calibration are weird. I'm using board 4x3, so it has 6 points. Log says that detector found 6 points in all images (134):
0000285 | 2023-11-09, 14:37:35.491338 [info] - Number of detected corners :: 6
0000286 | 2023-11-09, 14:37:35.526970 [info] - Number of detected corners :: 6
0000287 | 2023-11-09, 14:37:35.532342 [info] - Number of detected corners :: 6
0000288 | 2023-11-09, 14:37:35.725408 [info] - Board extraction done!
0000289 | 2023-11-09, 14:37:35.725435 [info] - Intrinsic calibration initiated
0000290 | 2023-11-09, 14:37:35.725441 [info] - Initializing camera calibration using images
0000291 | 2023-11-09, 14:37:35.725449 [info] - NB of board available in this camera :: 134
0000292 | 2023-11-09, 14:37:35.725457 [info] - NB of frames where this camera saw a board :: 134
0000293 | 2023-11-09, 14:37:52.858604 [info] - cameraMatrix : [730.5722449957763, 0, 1842.225576510645;
0, 103.8567137340529, 1010.620326430428;
0, 0, 1]
0000294 | 2023-11-09, 14:37:52.858662 [info] - distCoeffs : [-0.000145485069557112, -9.341887444497364e-07, -0.001914292531187723, -0.00490921035096062, -8.294359195743176e-08]
0000295 | 2023-11-09, 14:37:52.860456 [debug] - Trans :: [3.758818329998107;
7.510510097752508;
8.679588315941288] Rot :: [-1.192318438958881;
1.614630443992688;
-0.9228510025860922]
But then I see some calculations are done on different number of points than 6:
0005110 | 2023-11-09, 14:38:20.715560 [debug] - cam_id :: 0 object_id :: 0 mean error :: 29.6129 nb pts :: 3
0005111 | 2023-11-09, 14:38:20.715595 [debug] - cam_id :: 0 object_id :: 0 mean error :: 27.1408 nb pts :: 3
0005112 | 2023-11-09, 14:38:20.715629 [debug] - cam_id :: 0 object_id :: 0 mean error :: 81.2194 nb pts :: 3
0005113 | 2023-11-09, 14:38:20.715668 [debug] - cam_id :: 0 object_id :: 0 mean error :: 66.0296 nb pts :: 3
0005114 | 2023-11-09, 14:38:20.715703 [debug] - cam_id :: 0 object_id :: 0 mean error :: 88.8366 nb pts :: 3
0005115 | 2023-11-09, 14:38:20.715740 [debug] - cam_id :: 0 object_id :: 0 mean error :: 291.77 nb pts :: 3
0005116 | 2023-11-09, 14:38:20.715775 [debug] - cam_id :: 0 object_id :: 0 mean error :: 229.415 nb pts :: 3
0005117 | 2023-11-09, 14:38:20.715811 [debug] - cam_id :: 0 object_id :: 0 mean error :: 217.277 nb pts :: 3
0005118 | 2023-11-09, 14:38:20.715848 [debug] - cam_id :: 0 object_id :: 0 mean error :: 215.845 nb pts :: 3
0005119 | 2023-11-09, 14:38:20.715882 [debug] - cam_id :: 0 object_id :: 0 mean error :: 581.92 nb pts :: 4
0005120 | 2023-11-09, 14:38:20.715920 [debug] - cam_id :: 0 object_id :: 0 mean error :: 590.8 nb pts :: 4
0005121 | 2023-11-09, 14:38:20.715955 [debug] - cam_id :: 0 object_id :: 0 mean error :: 287.132 nb pts :: 3
0005122 | 2023-11-09, 14:38:20.715967 [info] - Final refinement done
0005123 | 2023-11-09, 14:38:38.345637 [info] - Save parameters
0005124 | 2023-11-09, 14:38:38.356766 [info] - mean reprojection error :: 175.409
0005125 | 2023-11-09, 14:38:38.360028 [info] - Calibration took 72 seconds
I'm using a subset size of 150 frames to get these results.
Sometimes the script produces some weird output like above, but more often it's failing due to this error:
0001480 | 2023-11-09, 14:56:08.856804 [info] - Frame :: 169 Boards :: 0 --- Mean Error ::12.0435 Nb pts :: 2
0001481 | 2023-11-09, 14:56:08.856821 [info] - High error detected in this board pose estimation
0001482 | 2023-11-09, 14:56:08.856827 [info] - Frame :: 161 Boards :: 0 --- Mean Error ::10.9782 Nb pts :: 2
0001483 | 2023-11-09, 14:56:08.856844 [info] - High error detected in this board pose estimation
0001484 | 2023-11-09, 14:56:08.856849 [info] - Frame :: 167 Boards :: 0 --- Mean Error ::12.6696 Nb pts :: 2
0001485 | 2023-11-09, 14:56:08.856867 [info] - High error detected in this board pose estimation
0001486 | 2023-11-09, 14:56:08.856872 [info] - Frame :: 168 Boards :: 0 --- Mean Error ::11.8517 Nb pts :: 2
0001487 | 2023-11-09, 14:56:08.856898 [info] - High error detected in this board pose estimation
0001488 | 2023-11-09, 14:56:08.856904 [info] - Frame :: 171 Boards :: 0 --- Mean Error ::10.6186 Nb pts :: 2
terminate called after throwing an instance of 'cv::Exception'
what(): OpenCV(4.2.0) ../modules/core/src/matrix_expressions.cpp:23: error: (-5:Bad argument) Matrix operand is an empty matrix. in function 'checkOperandsExist'
bash: line 1: 51 Aborted (core dumped) ./build/apps/calibrate/calibrate /home/sportv03/MC-Calib/data/subsets/subset_01/calib_param.yml
I don't know why. What am I doing wrong?
Does 4x3 charuco board is valid for this calibration? I know that 4 points are minimum for calculations, but maybe this number is higher for some reason?
Extrinsic calibration works on synthetic data on this 4x3 charuco board (intrinsic calibration is defined by me).
It would be great to have more configuration validation or data validation before each step of the script, so it would be more obvious to everyone why something is not working.
DawidStankiewicz commented
This image frameau/opencv-ceres seems old, I don't know why it was in the code, but when I switched to this bailool/mc-calib-prod, then I get same error I sometimes got before:
0000927 | 2023-11-09, 19:24:45.767402 [info] - High error detected in this board pose estimation
0000928 | 2023-11-09, 19:24:45.767409 [info] - Frame :: 141 Boards :: 0 --- Mean Error ::141.812 Nb pts :: 3
0000929 | 2023-11-09, 19:24:45.767429 [info] - High error detected in this board pose estimation
0000930 | 2023-11-09, 19:24:45.767435 [info] - Frame :: 146 Boards :: 0 --- Mean Error ::303.462 Nb pts :: 3
0000931 | 2023-11-09, 19:24:45.767473 [info] - High error detected in this board pose estimation
0000932 | 2023-11-09, 19:24:45.767480 [info] - Frame :: 147 Boards :: 0 --- Mean Error ::311.97 Nb pts :: 3
0000933 | 2023-11-09, 19:24:45.767500 [info] - High error detected in this board pose estimation
0000934 | 2023-11-09, 19:24:45.767506 [info] - Frame :: 149 Boards :: 0 --- Mean Error ::335.033 Nb pts :: 3
0000935 | 2023-11-09, 19:24:45.767529 [info] - High error detected in this board pose estimation
0000936 | 2023-11-09, 19:24:45.767535 [info] - Frame :: 148 Boards :: 0 --- Mean Error ::313.325 Nb pts :: 3
0000937 | 2023-11-09, 19:24:45.767543 [info] - Intrinsic Calibration done!
0000938 | 2023-11-09, 19:24:45.767549 [info] - 3D Object calibration initiated
=================================================================
==19==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x602000901b1c at pc 0x555e2e3ea891 bp 0x7ffeac988490 sp 0x7ffeac988480
READ of size 4 at 0x602000901b1c thread T0
#0 0x555e2e3ea890 in ransacP3P(std::vector<cv::Point3_<float>, std::allocator<cv::Point3_<float> > > const&, std::vector<cv::Point_<float>, std::allocator<cv::Point_<float> > > const&, cv::Mat, cv::Mat, cv::Mat&, cv::Mat&, float, int, double, bool) /home/sportv03/MC-Calib/McCalib/src/geometrytools.cpp:252
#1 0x555e2e3eceb8 in ransacP3PDistortion(std::vector<cv::Point3_<float>, std::allocator<cv::Point3_<float> > > const&, std::vector<cv::Point_<float>, std::allocator<cv::Point_<float> > > const&, cv::Mat, cv::Mat, cv::Mat&, cv::Mat&, float, int, int, double, bool) /home/sportv03/MC-Calib/McCalib/src/geometrytools.cpp:576
#2 0x555e2e3d4c5e in Object3DObs::estimatePose(float, int) /home/sportv03/MC-Calib/McCalib/src/Object3DObs.cpp:237
#3 0x555e2e289ee2 in McCalib::Calibration::estimatePoseAllObjects() /home/sportv03/MC-Calib/McCalib/src/McCalib.cpp:979
#4 0x555e2e2b5fc5 in McCalib::Calibration::calibrate3DObjects() /home/sportv03/MC-Calib/McCalib/src/McCalib.cpp:2084
#5 0x555e2e2301c5 in runCalibrationWorkflow(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >) /home/sportv03/MC-Calib/apps/calibrate/src/calibrate.cpp:21
#6 0x555e2e225c15 in main /home/sportv03/MC-Calib/apps/calibrate/src/calibrate.cpp:97
#7 0x7f9ad5c95082 in __libc_start_main (/usr/lib/x86_64-linux-gnu/libc.so.6+0x24082)
#8 0x555e2e22dcdd in _start (/home/sportv03/MC-Calib/build/apps/calibrate/calibrate+0xb7cdd)
0x602000901b1c is located 0 bytes to the right of 12-byte region [0x602000901b10,0x602000901b1c)
allocated by thread T0 here:
#0 0x7f9ad8f4f587 in operator new(unsigned long) ../../../../src/libsanitizer/asan/asan_new_delete.cc:104
#1 0x555e2e3e5949 in __gnu_cxx::new_allocator<int>::allocate(unsigned long, void const*) /usr/include/c++/9/ext/new_allocator.h:114
#2 0x555e2e3e5949 in std::allocator_traits<std::allocator<int> >::allocate(std::allocator<int>&, unsigned long) /usr/include/c++/9/bits/alloc_traits.h:443
#3 0x555e2e3e5949 in std::_Vector_base<int, std::allocator<int> >::_M_allocate(unsigned long) /usr/include/c++/9/bits/stl_vector.h:343
#4 0x555e2e3e5949 in std::_Vector_base<int, std::allocator<int> >::_M_create_storage(unsigned long) /usr/include/c++/9/bits/stl_vector.h:358
#5 0x555e2e3e5949 in std::_Vector_base<int, std::allocator<int> >::_Vector_base(unsigned long, std::allocator<int> const&) /usr/include/c++/9/bits/stl_vector.h:302
#6 0x555e2e3e5949 in std::vector<int, std::allocator<int> >::vector(unsigned long, std::allocator<int> const&) /usr/include/c++/9/bits/stl_vector.h:508
#7 0x555e2e3e5949 in ransacP3P(std::vector<cv::Point3_<float>, std::allocator<cv::Point3_<float> > > const&, std::vector<cv::Point_<float>, std::allocator<cv::Point_<float> > > const&, cv::Mat, cv::Mat, cv::Mat&, cv::Mat&, float, int, double, bool) /home/sportv03/MC-Calib/McCalib/src/geometrytools.cpp:243
#8 0x555e2e3eceb8 in ransacP3PDistortion(std::vector<cv::Point3_<float>, std::allocator<cv::Point3_<float> > > const&, std::vector<cv::Point_<float>, std::allocator<cv::Point_<float> > > const&, cv::Mat, cv::Mat, cv::Mat&, cv::Mat&, float, int, int, double, bool) /home/sportv03/MC-Calib/McCalib/src/geometrytools.cpp:576
#9 0x555e2e3d4c5e in Object3DObs::estimatePose(float, int) /home/sportv03/MC-Calib/McCalib/src/Object3DObs.cpp:237
#10 0x555e2e289ee2 in McCalib::Calibration::estimatePoseAllObjects() /home/sportv03/MC-Calib/McCalib/src/McCalib.cpp:979
SUMMARY: AddressSanitizer: heap-buffer-overflow /home/sportv03/MC-Calib/McCalib/src/geometrytools.cpp:252 in ransacP3P(std::vector<cv::Point3_<float>, std::allocator<cv::Point3_<float> > > const&, std::vector<cv::Point_<float>, std::allocator<cv::Point_<float> > > const&, cv::Mat, cv::Mat, cv::Mat&, cv::Mat&, float, int, double, bool)
Shadow bytes around the buggy address:
0x0c0480118310: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x0c0480118320: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x0c0480118330: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x0c0480118340: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x0c0480118350: fa fa fa fa fa fa 00 04 fa fa 04 fa fa fa 04 fa
=>0x0c0480118360: fa fa 00[04]fa fa fa fa fa fa fa fa fa fa fa fa
0x0c0480118370: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x0c0480118380: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x0c0480118390: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x0c04801183a0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x0c04801183b0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
Shadow byte legend (one shadow byte represents 8 application bytes):
Addressable: 00
Partially addressable: 01 02 03 04 05 06 07
Heap left redzone: fa
Freed heap region: fd
Stack left redzone: f1
Stack mid redzone: f2
Stack right redzone: f3
Stack after return: f5
Stack use after scope: f8
Global redzone: f9
Global init order: f6
Poisoned by user: f7
Container overflow: fc
Array cookie: ac
Intra object redzone: bb
ASan internal: fe
Left alloca redzone: ca
Right alloca redzone: cb
Shadow gap: cc
==19==ABORTING
If it's geometrytools.cpp:252 then it looks like a problem with not enough points 🤔 but still log says that on all images detected 6 points.
// pick 4 points
std::random_shuffle(myvector.begin(), myvector.end());
std::array<int, 4> idx = {myvector[0], myvector[1], myvector[2],
myvector[3]};DawidStankiewicz commented
Ok it was simpler than it looked. Here is the problem:
######################################## Boards Parameters ###################################################
number_x_square: 4 # number of squares in the X direction
number_y_square: 3 # number of squares the Y direction
after switching to 3x4 it's working
######################################## Boards Parameters ###################################################
number_x_square: 3 # number of squares in the X direction
number_y_square: 4 # number of squares the Y directionSo it's my fault, I printed the board with wrong parameters.
Is it possible to add some warnings or error messages to prevent mistakes like this?
DawidStankiewicz commented
Leaving the trace, here is my correct (I guess) output cameras.yml:
%YAML:1.0
---
nb_camera: 1
camera_0:
camera_matrix: !!opencv-matrix
rows: 3
cols: 3
dt: d
data: [ 3.4089219026871656e+03, 0., 1.9197051479126044e+03, 0.,
3.4123589126351535e+03, 1.0866674255497603e+03, 0., 0., 1. ]
distortion_vector: !!opencv-matrix
rows: 1
cols: 5
dt: d
data: [ -2.3354408055534709e-02, -1.8428057118907593e-02,
3.5474013129843543e-04, 8.8942147821997972e-04,
2.5178062482779479e-01 ]
distortion_type: 0
camera_group: 0
img_width: 3840
img_height: 2160
camera_pose_matrix: !!opencv-matrix
rows: 4
cols: 4
dt: d
data: [ 1., 0., 0., 0., 0., 1., 0., 0., 0., 0., 1., 0., 0., 0., 0.,
1. ]
errors:
camera_0:
translation_std: !!opencv-matrix
rows: 3
cols: 1
dt: d
data: [ 0., 0., 0. ]
mean_rotation_error: 0.using this configration
boards_index: []
cam_params_path: None
cam_prefix: Cam_
camera_params_file_name: cameras.yml
distortion_model: 0
distortion_per_camera: []
fix_intrinsic: 0
he_approach: 0
length_marker: 0.03
length_square: 0.04
min_perc_pts: 0.8
number_board: 1
number_camera: 1
number_iterations: 1000
number_x_square: 3
number_x_square_per_board: []
number_y_square: 4
number_y_square_per_board: []
ransac_threshold: 5
refine_corner: 1
resolution_x: 3840
resolution_y: 2160
save_detection: 1
save_reprojection: 1
square_size: 6.5
square_size_per_board: []Example frame with detections:
And here is full log of this calibration:
0000001 | 2023-11-09, 19:41:55.931742 [info] - Nb of cameras : 1 Nb of Boards : 1 Refined Corners : true Distortion mode : 0
0000002 | 2023-11-09, 19:41:55.932443 [info] - Extraction camera 001
0000003 | 2023-11-09, 19:41:55.954703 [info] - Number of threads for board detection :: 20
0000154 | 2023-11-09, 19:41:56.391658 [info] - Number of detected corners :: 6
0000155 | 2023-11-09, 19:41:56.883855 [info] - Number of detected corners :: 6
0000156 | 2023-11-09, 19:41:57.014276 [info] - Number of detected corners :: 6
0000157 | 2023-11-09, 19:41:57.025579 [info] - Number of detected corners :: 6
0000158 | 2023-11-09, 19:41:57.035072 [info] - Number of detected corners :: 6
0000159 | 2023-11-09, 19:41:57.043034 [info] - Number of detected corners :: 6
0000160 | 2023-11-09, 19:41:57.046643 [info] - Number of detected corners :: 6
0000161 | 2023-11-09, 19:41:57.051452 [info] - Number of detected corners :: 6
0000162 | 2023-11-09, 19:41:57.052213 [info] - Number of detected corners :: 6
0000163 | 2023-11-09, 19:41:57.064255 [info] - Number of detected corners :: 6
0000164 | 2023-11-09, 19:41:57.067523 [info] - Number of detected corners :: 6
0000165 | 2023-11-09, 19:41:57.069675 [info] - Number of detected corners :: 6
0000166 | 2023-11-09, 19:41:57.071843 [info] - Number of detected corners :: 6
0000167 | 2023-11-09, 19:41:57.072352 [info] - Number of detected corners :: 6
0000168 | 2023-11-09, 19:41:57.074574 [info] - Number of detected corners :: 6
0000169 | 2023-11-09, 19:41:57.078415 [info] - Number of detected corners :: 6
0000170 | 2023-11-09, 19:41:57.078799 [info] - Number of detected corners :: 6
0000171 | 2023-11-09, 19:41:57.081711 [info] - Number of detected corners :: 6
0000172 | 2023-11-09, 19:41:57.187564 [info] - Number of detected corners :: 6
0000173 | 2023-11-09, 19:41:57.788288 [info] - Number of detected corners :: 6
0000174 | 2023-11-09, 19:41:57.846747 [info] - Number of detected corners :: 6
0000175 | 2023-11-09, 19:41:57.915139 [info] - Number of detected corners :: 6
0000176 | 2023-11-09, 19:41:58.377457 [info] - Number of detected corners :: 6
0000177 | 2023-11-09, 19:41:58.408988 [info] - Number of detected corners :: 6
0000178 | 2023-11-09, 19:41:58.417557 [info] - Number of detected corners :: 6
0000179 | 2023-11-09, 19:41:58.420223 [info] - Number of detected corners :: 6
0000180 | 2023-11-09, 19:41:58.422251 [info] - Number of detected corners :: 6
0000181 | 2023-11-09, 19:41:58.422873 [info] - Number of detected corners :: 6
0000182 | 2023-11-09, 19:41:58.424522 [info] - Number of detected corners :: 6
0000183 | 2023-11-09, 19:41:58.430306 [info] - Number of detected corners :: 6
0000184 | 2023-11-09, 19:41:58.435548 [info] - Number of detected corners :: 6
0000185 | 2023-11-09, 19:41:58.435657 [info] - Number of detected corners :: 6
0000186 | 2023-11-09, 19:41:58.439173 [info] - Number of detected corners :: 6
0000187 | 2023-11-09, 19:41:58.442691 [info] - Number of detected corners :: 6
0000188 | 2023-11-09, 19:41:58.443259 [info] - Number of detected corners :: 6
0000189 | 2023-11-09, 19:41:58.456990 [info] - Number of detected corners :: 6
0000190 | 2023-11-09, 19:41:58.468937 [info] - Number of detected corners :: 6
0000191 | 2023-11-09, 19:41:58.602380 [info] - Number of detected corners :: 6
0000192 | 2023-11-09, 19:41:59.179171 [info] - Number of detected corners :: 6
0000193 | 2023-11-09, 19:41:59.181499 [info] - Number of detected corners :: 6
0000194 | 2023-11-09, 19:41:59.230882 [info] - Number of detected corners :: 6
0000195 | 2023-11-09, 19:41:59.294396 [info] - Number of detected corners :: 6
0000196 | 2023-11-09, 19:41:59.616744 [info] - Number of detected corners :: 6
0000197 | 2023-11-09, 19:41:59.763195 [info] - Number of detected corners :: 6
0000198 | 2023-11-09, 19:41:59.772432 [info] - Number of detected corners :: 6
0000199 | 2023-11-09, 19:41:59.779965 [info] - Number of detected corners :: 6
0000200 | 2023-11-09, 19:41:59.791094 [info] - Number of detected corners :: 6
0000201 | 2023-11-09, 19:41:59.800951 [info] - Number of detected corners :: 6
0000202 | 2023-11-09, 19:41:59.804885 [info] - Number of detected corners :: 6
0000203 | 2023-11-09, 19:41:59.813097 [info] - Number of detected corners :: 6
0000204 | 2023-11-09, 19:41:59.820213 [info] - Number of detected corners :: 6
0000205 | 2023-11-09, 19:41:59.831461 [info] - Number of detected corners :: 6
0000206 | 2023-11-09, 19:41:59.856650 [info] - Number of detected corners :: 6
0000207 | 2023-11-09, 19:41:59.869923 [info] - Number of detected corners :: 6
0000208 | 2023-11-09, 19:41:59.985144 [info] - Number of detected corners :: 6
0000209 | 2023-11-09, 19:42:00.007011 [info] - Number of detected corners :: 6
0000210 | 2023-11-09, 19:42:00.028917 [info] - Number of detected corners :: 6
0000211 | 2023-11-09, 19:42:00.053372 [info] - Number of detected corners :: 6
0000212 | 2023-11-09, 19:42:00.190402 [info] - Number of detected corners :: 6
0000213 | 2023-11-09, 19:42:00.604648 [info] - Number of detected corners :: 6
0000214 | 2023-11-09, 19:42:00.696202 [info] - Number of detected corners :: 6
0000215 | 2023-11-09, 19:42:00.751165 [info] - Number of detected corners :: 6
0000216 | 2023-11-09, 19:42:00.817243 [info] - Number of detected corners :: 6
0000217 | 2023-11-09, 19:42:01.126580 [info] - Number of detected corners :: 6
0000218 | 2023-11-09, 19:42:01.163259 [info] - Number of detected corners :: 6
0000219 | 2023-11-09, 19:42:01.237139 [info] - Number of detected corners :: 6
0000220 | 2023-11-09, 19:42:01.263114 [info] - Number of detected corners :: 6
0000221 | 2023-11-09, 19:42:01.280151 [info] - Number of detected corners :: 6
0000222 | 2023-11-09, 19:42:01.286722 [info] - Number of detected corners :: 6
0000223 | 2023-11-09, 19:42:01.301338 [info] - Number of detected corners :: 6
0000224 | 2023-11-09, 19:42:01.305156 [info] - Number of detected corners :: 6
0000225 | 2023-11-09, 19:42:01.325084 [info] - Number of detected corners :: 6
0000226 | 2023-11-09, 19:42:01.484580 [info] - Number of detected corners :: 6
0000227 | 2023-11-09, 19:42:01.517241 [info] - Number of detected corners :: 6
0000228 | 2023-11-09, 19:42:01.528997 [info] - Number of detected corners :: 6
0000229 | 2023-11-09, 19:42:01.536519 [info] - Number of detected corners :: 6
0000230 | 2023-11-09, 19:42:01.541458 [info] - Number of detected corners :: 6
0000231 | 2023-11-09, 19:42:01.619411 [info] - Number of detected corners :: 6
0000232 | 2023-11-09, 19:42:02.040156 [info] - Number of detected corners :: 6
0000233 | 2023-11-09, 19:42:02.216112 [info] - Number of detected corners :: 6
0000234 | 2023-11-09, 19:42:02.234455 [info] - Number of detected corners :: 6
0000235 | 2023-11-09, 19:42:02.240154 [info] - Number of detected corners :: 6
0000236 | 2023-11-09, 19:42:02.256737 [info] - Number of detected corners :: 6
0000237 | 2023-11-09, 19:42:02.271854 [info] - Number of detected corners :: 6
0000238 | 2023-11-09, 19:42:02.380276 [info] - Number of detected corners :: 6
0000239 | 2023-11-09, 19:42:02.603583 [info] - Number of detected corners :: 6
0000240 | 2023-11-09, 19:42:02.612805 [info] - Number of detected corners :: 6
0000241 | 2023-11-09, 19:42:02.662214 [info] - Number of detected corners :: 6
0000242 | 2023-11-09, 19:42:02.701055 [info] - Number of detected corners :: 6
0000243 | 2023-11-09, 19:42:02.711006 [info] - Number of detected corners :: 6
0000244 | 2023-11-09, 19:42:02.737259 [info] - Number of detected corners :: 6
0000245 | 2023-11-09, 19:42:02.760583 [info] - Number of detected corners :: 6
0000246 | 2023-11-09, 19:42:02.950104 [info] - Number of detected corners :: 6
0000247 | 2023-11-09, 19:42:02.951831 [info] - Number of detected corners :: 6
0000248 | 2023-11-09, 19:42:02.967098 [info] - Number of detected corners :: 6
0000249 | 2023-11-09, 19:42:02.981049 [info] - Number of detected corners :: 6
0000250 | 2023-11-09, 19:42:03.077919 [info] - Number of detected corners :: 6
0000251 | 2023-11-09, 19:42:03.459420 [info] - Number of detected corners :: 6
0000252 | 2023-11-09, 19:42:03.689569 [info] - Number of detected corners :: 6
0000253 | 2023-11-09, 19:42:03.694179 [info] - Number of detected corners :: 6
0000254 | 2023-11-09, 19:42:03.704636 [info] - Number of detected corners :: 6
0000255 | 2023-11-09, 19:42:03.704853 [info] - Number of detected corners :: 6
0000256 | 2023-11-09, 19:42:03.705908 [info] - Number of detected corners :: 6
0000257 | 2023-11-09, 19:42:03.740245 [info] - Number of detected corners :: 6
0000258 | 2023-11-09, 19:42:03.753769 [info] - Number of detected corners :: 6
0000259 | 2023-11-09, 19:42:03.945341 [info] - Number of detected corners :: 6
0000260 | 2023-11-09, 19:42:04.002475 [info] - Number of detected corners :: 6
0000261 | 2023-11-09, 19:42:04.127464 [info] - Number of detected corners :: 6
0000262 | 2023-11-09, 19:42:04.153221 [info] - Number of detected corners :: 6
0000263 | 2023-11-09, 19:42:04.229629 [info] - Number of detected corners :: 6
0000264 | 2023-11-09, 19:42:04.284294 [info] - Number of detected corners :: 6
0000265 | 2023-11-09, 19:42:04.340753 [info] - Number of detected corners :: 6
0000266 | 2023-11-09, 19:42:04.530619 [info] - Number of detected corners :: 6
0000267 | 2023-11-09, 19:42:04.685421 [info] - Number of detected corners :: 6
0000268 | 2023-11-09, 19:42:04.897467 [info] - Number of detected corners :: 6
0000269 | 2023-11-09, 19:42:05.225240 [info] - Number of detected corners :: 6
0000270 | 2023-11-09, 19:42:05.255726 [info] - Number of detected corners :: 6
0000271 | 2023-11-09, 19:42:05.274892 [info] - Number of detected corners :: 6
0000272 | 2023-11-09, 19:42:05.276763 [info] - Number of detected corners :: 6
0000273 | 2023-11-09, 19:42:05.278098 [info] - Number of detected corners :: 6
0000274 | 2023-11-09, 19:42:05.286148 [info] - Number of detected corners :: 6
0000275 | 2023-11-09, 19:42:05.300776 [info] - Number of detected corners :: 6
0000276 | 2023-11-09, 19:42:05.372645 [info] - Number of detected corners :: 6
0000277 | 2023-11-09, 19:42:05.402864 [info] - Number of detected corners :: 6
0000278 | 2023-11-09, 19:42:05.423072 [info] - Number of detected corners :: 6
0000279 | 2023-11-09, 19:42:05.552719 [info] - Number of detected corners :: 6
0000280 | 2023-11-09, 19:42:05.555362 [info] - Number of detected corners :: 6
0000281 | 2023-11-09, 19:42:05.565536 [info] - Number of detected corners :: 6
0000282 | 2023-11-09, 19:42:05.713833 [info] - Number of detected corners :: 6
0000283 | 2023-11-09, 19:42:05.816718 [info] - Number of detected corners :: 6
0000284 | 2023-11-09, 19:42:06.029115 [info] - Board extraction done!
0000285 | 2023-11-09, 19:42:06.029148 [info] - Intrinsic calibration initiated
0000286 | 2023-11-09, 19:42:06.029156 [info] - Initializing camera calibration using images
0000287 | 2023-11-09, 19:42:06.029161 [info] - NB of board available in this camera :: 130
0000288 | 2023-11-09, 19:42:06.029172 [info] - NB of frames where this camera saw a board :: 130
0000289 | 2023-11-09, 19:42:18.670284 [info] - cameraMatrix : [3405.219129797035, 0, 1910.094988443297;
0, 3410.201075447987, 1117.978734413974;
0, 0, 1]
0000290 | 2023-11-09, 19:42:18.670343 [info] - distCoeffs : [0.03576338019176255, -0.3691663487185353, 0.002828214851890153, -0.0002464721235971383, 1.005100191367455]
0000681 | 2023-11-09, 19:42:18.695245 [info] - Parameters before optimization :: [3405.219129797035, 0, 1910.094988443297;
0, 3410.201075447987, 1117.978734413974;
0, 0, 1]
0000682 | 2023-11-09, 19:42:18.695289 [info] - distortion vector :: [0.03576338019176255, -0.3691663487185353, 0.002828214851890153, -0.0002464721235971383, 1.005100191367455]
iter cost cost_change |gradient| |step| tr_ratio tr_radius ls_iter iter_time total_time
0 7.290947e+01 0.00e+00 1.62e+03 0.00e+00 0.00e+00 1.00e+04 0 1.16e-02 1.26e-02
1 6.926904e+01 3.64e+00 1.25e+03 3.05e+00 1.04e+00 3.00e+04 1 1.15e-02 2.41e-02
2 6.782618e+01 1.44e+00 1.46e+02 1.64e+00 1.04e+00 9.00e+04 1 1.19e-02 3.61e-02
3 6.735006e+01 4.76e-01 1.85e+02 3.37e+00 1.07e+00 2.70e+05 1 1.23e-02 4.84e-02
4 6.707892e+01 2.71e-01 1.99e+02 6.53e+00 1.02e+00 8.10e+05 1 1.15e-02 5.99e-02
5 6.680882e+01 2.70e-01 6.25e+02 1.06e+01 9.51e-01 2.43e+06 1 1.17e-02 7.17e-02
6 6.668359e+01 1.25e-01 5.10e+02 9.81e+00 9.32e-01 6.82e+06 1 1.22e-02 8.39e-02
7 6.666008e+01 2.35e-02 6.85e+01 3.64e+00 1.05e+00 2.05e+07 1 1.15e-02 9.54e-02
8 6.665849e+01 1.59e-03 8.33e+00 5.95e-01 1.40e+00 6.14e+07 1 1.15e-02 1.07e-01
9 6.665805e+01 4.35e-04 3.82e+00 1.03e-01 1.62e+00 1.84e+08 1 1.22e-02 1.19e-01
10 6.665788e+01 1.72e-04 2.08e+00 3.96e-02 1.64e+00 5.53e+08 1 1.23e-02 1.31e-01
11 6.665781e+01 7.05e-05 1.25e+00 2.13e-02 1.64e+00 1.66e+09 1 1.16e-02 1.43e-01
0000683 | 2023-11-09, 19:42:18.841165 [info] - Parameters after optimization :: [3408.921902687166, 0, 1919.705147912604;
0, 3412.358912635154, 1086.66742554976;
0, 0, 1]
0000684 | 2023-11-09, 19:42:18.841210 [info] - distortion vector after optimization :: [-0.02335440805553471, -0.01842805711890759, 0.0003547401312984354, 0.0008894214782199797, 0.2517806248277948]
0000685 | 2023-11-09, 19:42:18.843411 [info] - Intrinsic Calibration done!
0000686 | 2023-11-09, 19:42:18.843422 [info] - 3D Object calibration initiated
0001211 | 2023-11-09, 19:42:18.868771 [info] - Mean Error 0.351087
iter cost cost_change |gradient| |step| tr_ratio tr_radius ls_iter iter_time total_time
0 6.747345e+01 0.00e+00 3.39e+02 0.00e+00 0.00e+00 1.00e+04 0 7.18e-03 8.04e-03
1 6.676715e+01 7.06e-01 2.11e+02 1.04e-01 1.34e+00 3.00e+04 1 8.07e-03 1.61e-02
2 6.667512e+01 9.20e-02 3.11e+01 1.27e-02 1.17e+00 9.00e+04 1 8.78e-03 2.49e-02
3 6.666652e+01 8.60e-03 2.49e+01 3.37e-03 1.60e+00 2.70e+05 1 7.91e-03 3.28e-02
4 6.666276e+01 3.76e-03 2.00e+01 3.15e-03 1.72e+00 8.10e+05 1 7.81e-03 4.07e-02
5 6.666071e+01 2.05e-03 1.61e+01 2.24e-03 1.77e+00 2.43e+06 1 7.82e-03 4.85e-02
6 6.665944e+01 1.27e-03 1.30e+01 1.63e-03 1.81e+00 7.29e+06 1 8.64e-03 5.71e-02
7 6.665867e+01 7.70e-04 1.05e+01 1.26e-03 1.63e+00 2.19e+07 1 7.87e-03 6.50e-02
8 6.665830e+01 3.71e-04 8.49e+00 7.35e-04 1.74e+00 6.56e+07 1 7.80e-03 7.28e-02
9 6.665809e+01 2.07e-04 6.88e+00 5.13e-04 1.76e+00 1.97e+08 1 7.94e-03 8.08e-02
10 6.665797e+01 1.21e-04 5.58e+00 3.71e-04 1.78e+00 5.90e+08 1 8.74e-03 8.95e-02
11 6.665789e+01 7.38e-05 4.53e+00 2.75e-04 1.79e+00 1.77e+09 1 8.35e-03 9.79e-02
0001342 | 2023-11-09, 19:42:18.972266 [info] - Mean Error 0.349113
0001343 | 2023-11-09, 19:42:18.972296 [info] - 3D Object calibration done!
0001344 | 2023-11-09, 19:42:18.972301 [info] - Camera group calibration initiated
0001348 | 2023-11-09, 19:42:18.975133 [info] - Number of frames for camera group optimization :: 130
iter cost cost_change |gradient| |step| tr_ratio tr_radius ls_iter iter_time total_time
0 6.665789e+01 0.00e+00 4.53e+00 0.00e+00 0.00e+00 1.00e+04 0 1.06e-02 1.17e-02
0001349 | 2023-11-09, 19:42:18.990538 [info] - Camera 0 :: [1, 0, 0, 0;
0, 1, 0, 0;
0, 0, 1, 0;
0, 0, 0, 1]
0001350 | 2023-11-09, 19:42:18.991763 [info] - Camera group calibration done!
0002262 | 2023-11-09, 19:42:19.048723 [info] - Number of frames for camera group optimization :: 130
iter cost cost_change |gradient| |step| tr_ratio tr_radius ls_iter iter_time total_time
0 6.732523e+01 0.00e+00 3.39e+02 0.00e+00 0.00e+00 1.00e+04 0 9.89e-03 1.08e-02
1 6.676684e+01 5.58e-01 2.11e+02 4.65e-02 1.46e+00 3.00e+04 1 1.18e-02 2.27e-02
2 6.667512e+01 9.17e-02 3.11e+01 1.21e-02 1.17e+00 9.00e+04 1 1.11e-02 3.38e-02
3 6.666652e+01 8.60e-03 2.49e+01 3.36e-03 1.60e+00 2.70e+05 1 1.17e-02 4.55e-02
4 6.666276e+01 3.76e-03 2.00e+01 3.15e-03 1.72e+00 8.10e+05 1 1.09e-02 5.65e-02
5 6.666071e+01 2.05e-03 1.61e+01 2.24e-03 1.77e+00 2.43e+06 1 1.09e-02 6.74e-02
6 6.665944e+01 1.27e-03 1.30e+01 1.63e-03 1.81e+00 7.29e+06 1 1.16e-02 7.89e-02
7 6.665867e+01 7.70e-04 1.05e+01 1.26e-03 1.63e+00 2.19e+07 1 1.10e-02 8.99e-02
8 6.665830e+01 3.71e-04 8.49e+00 7.35e-04 1.74e+00 6.56e+07 1 1.17e-02 1.02e-01
9 6.665809e+01 2.07e-04 6.88e+00 5.13e-04 1.76e+00 1.97e+08 1 1.09e-02 1.13e-01
10 6.665797e+01 1.21e-04 5.58e+00 3.71e-04 1.78e+00 5.90e+08 1 1.15e-02 1.24e-01
11 6.665789e+01 7.38e-05 4.53e+00 2.75e-04 1.79e+00 1.77e+09 1 1.09e-02 1.35e-01
0002263 | 2023-11-09, 19:42:19.187167 [info] - Camera 0 :: [1, 0, 0, 0;
0, 1, 0, 0;
0, 0, 1, 0;
0, 0, 0, 1]
0002394 | 2023-11-09, 19:42:19.191057 [info] - Non-overlapping calibration initiated
0002395 | 2023-11-09, 19:42:19.191075 [info] - Non-overlapping calibration done!
0002396 | 2023-11-09, 19:42:19.191081 [info] - Merge cameras and objets initiated
0003308 | 2023-11-09, 19:42:19.246652 [info] - Number of frames for camera group optimization :: 130
iter cost cost_change |gradient| |step| tr_ratio tr_radius ls_iter iter_time total_time
0 6.732523e+01 0.00e+00 3.39e+02 0.00e+00 0.00e+00 1.00e+04 0 9.97e-03 1.09e-02
1 6.676684e+01 5.58e-01 2.11e+02 4.65e-02 1.46e+00 3.00e+04 1 1.12e-02 2.21e-02
2 6.667512e+01 9.17e-02 3.11e+01 1.21e-02 1.17e+00 9.00e+04 1 1.18e-02 3.39e-02
3 6.666652e+01 8.60e-03 2.49e+01 3.36e-03 1.60e+00 2.70e+05 1 1.17e-02 4.56e-02
4 6.666276e+01 3.76e-03 2.00e+01 3.15e-03 1.72e+00 8.10e+05 1 1.10e-02 5.66e-02
5 6.666071e+01 2.05e-03 1.61e+01 2.24e-03 1.77e+00 2.43e+06 1 1.16e-02 6.82e-02
6 6.665944e+01 1.27e-03 1.30e+01 1.63e-03 1.81e+00 7.29e+06 1 1.09e-02 7.91e-02
7 6.665867e+01 7.70e-04 1.05e+01 1.26e-03 1.63e+00 2.19e+07 1 1.08e-02 8.99e-02
8 6.665830e+01 3.71e-04 8.49e+00 7.35e-04 1.74e+00 6.56e+07 1 1.72e-02 1.07e-01
9 6.665809e+01 2.07e-04 6.88e+00 5.13e-04 1.76e+00 1.97e+08 1 1.08e-02 1.18e-01
10 6.665797e+01 1.21e-04 5.58e+00 3.71e-04 1.78e+00 5.90e+08 1 1.16e-02 1.30e-01
11 6.665789e+01 7.38e-05 4.53e+00 2.75e-04 1.79e+00 1.77e+09 1 1.09e-02 1.41e-01
0003309 | 2023-11-09, 19:42:19.390205 [info] - Camera 0 :: [1, 0, 0, 0;
0, 1, 0, 0;
0, 0, 1, 0;
0, 0, 0, 1]
0003830 | 2023-11-09, 19:42:19.420297 [info] - Merge cameras and objets done!
0003831 | 2023-11-09, 19:42:19.420325 [info] - Final refinement initiated
0003832 | 2023-11-09, 19:42:19.420334 [info] - Number of frames for camera group optimization :: 130
iter cost cost_change |gradient| |step| tr_ratio tr_radius ls_iter iter_time total_time
0 6.732523e+01 0.00e+00 3.39e+02 0.00e+00 0.00e+00 1.00e+04 0 9.94e-03 1.08e-02
1 6.676684e+01 5.58e-01 2.11e+02 4.65e-02 1.46e+00 3.00e+04 1 1.17e-02 2.26e-02
2 6.667512e+01 9.17e-02 3.11e+01 1.21e-02 1.17e+00 9.00e+04 1 1.09e-02 3.35e-02
3 6.666652e+01 8.60e-03 2.49e+01 3.36e-03 1.60e+00 2.70e+05 1 1.08e-02 4.43e-02
4 6.666276e+01 3.76e-03 2.00e+01 3.15e-03 1.72e+00 8.10e+05 1 1.15e-02 5.58e-02
5 6.666071e+01 2.05e-03 1.61e+01 2.24e-03 1.77e+00 2.43e+06 1 1.08e-02 6.66e-02
6 6.665944e+01 1.27e-03 1.30e+01 1.63e-03 1.81e+00 7.29e+06 1 1.08e-02 7.75e-02
7 6.665867e+01 7.70e-04 1.05e+01 1.26e-03 1.63e+00 2.19e+07 1 1.15e-02 8.90e-02
8 6.665830e+01 3.71e-04 8.49e+00 7.35e-04 1.74e+00 6.56e+07 1 1.08e-02 9.98e-02
9 6.665809e+01 2.07e-04 6.88e+00 5.13e-04 1.76e+00 1.97e+08 1 1.07e-02 1.11e-01
10 6.665797e+01 1.21e-04 5.58e+00 3.71e-04 1.78e+00 5.90e+08 1 1.17e-02 1.22e-01
11 6.665789e+01 7.38e-05 4.53e+00 2.75e-04 1.79e+00 1.77e+09 1 1.08e-02 1.33e-01
0003833 | 2023-11-09, 19:42:19.556461 [info] - Camera 0 :: [1, 0, 0, 0;
0, 1, 0, 0;
0, 0, 1, 0;
0, 0, 0, 1]
0003834 | 2023-11-09, 19:42:19.557965 [info] - Number of frames for camera group optimization :: 130
iter cost cost_change |gradient| |step| tr_ratio tr_radius ls_iter iter_time total_time
0 6.665789e+01 0.00e+00 4.53e+00 0.00e+00 0.00e+00 1.00e+04 0 2.10e-02 2.21e-02
0003835 | 2023-11-09, 19:42:19.584015 [info] - Camera 0 :: [1, 0, 0, 0;
0, 1, 0, 0;
0, 0, 1, 0;
0, 0, 0, 1]
0003966 | 2023-11-09, 19:42:19.587718 [info] - Final refinement done
0003967 | 2023-11-09, 19:42:36.831521 [info] - Save parameters
0003968 | 2023-11-09, 19:42:36.842377 [info] - mean reprojection error :: 0.349113
0003969 | 2023-11-09, 19:42:36.849276 [info] - Calibration took 40 seconds
=================================================================
==19==ERROR: LeakSanitizer: detected memory leaks
Direct leak of 12312 byte(s) in 3 object(s) allocated from:
#0 0x7f700471a787 in operator new[](unsigned long) ../../../../src/libsanitizer/asan/asan_new_delete.cc:107
#1 0x7f700244066f (/usr/lib/x86_64-linux-gnu/libtbb.so.2+0x2766f)
SUMMARY: AddressSanitizer: 12312 byte(s) leaked in 3 allocation(s).
rameau-fr commented
Thank you so much for the very detailed feedback; I am sure it will be useful for future users facing a similar issue. I am also glad you managed to find the mistake and calibrate your camera properly.
The idea of warning the user that the used pattern does not exhibit the proper configuration is a great idea and might be implemented quite easily. Unfortunately, I cannot promise such an update rapidly.
Thank you very much again!