Extrinsic parameters are off/wrong?
Closed this issue · 10 comments
Hi, I have been trying to use the dataset for a couple weeks and I have a problem with understanding how the extrinsic parameters are described, in particular the camera to IMU parameters seems off. The dataset I'm looking at in particular is HK-medium and I'll paste the extrinsics them below:
################## Extrinsic parameter between IMU and Camera##################
################## camera is ZED2, a stereo camera##################
LEFT_CAMERA_T_IMU:!!opencv-matrix
rows: 4
cols: 4
dt: d
data: [0.99958975976322017672, 0.017170804625958683, -0.022923255549840760448, -0.0851835233537896739,
0.0231465893493554992, -0.01292276982900426, 0.99964855695461152696, 0.1257734831824673213,
0.016868538110887141801, -0.99976905607667321087, -0.013314913952324877203, 0.075888549707280005484,
0.0, 0.0, 0.0, 1.0]
RIGHT_CAMERA_T_IMU:!!opencv-matrix
rows: 4
cols: 4
dt: d
data: [ 9.9971618725690459e-01, 9.9454436695301657e-03, -2.1647934930009227e-02, 2.1160626565556342e-01,
2.1740820070310918e-02, -9.3282535133940581e-03, 9.9972012104841623e-01, 1.5171555107814688e-01,
9.7407227241143209e-03, -9.9990703159674099e-01, -9.5418281375148073e-03, -1.9140752384352994e-01,
0., 0., 0., 1. ]
By comparing the schematics to the transformations, the rotations seems correct, however the translations does not.
In the case LEFT_CAMERA_T_IMU the camera is moved -8.5 cm along the x-axis in the IMU coordinate system and RIGHT_CAMERA_T_IMU is moved 21.1 cm along the same axis. This is the confusing part, since the cameras are zed2 cameras, they are enclosed in a frame with a distance of 12cm between the cameras. It seems that the RIGHT_CAMERA_T_IMU is not actually for the zed2 right camera but for the Monocular right camera as shown in the schematics (https://github.com/IPNL-POLYU/UrbanNavDataset).
Where the translation from the Monocular right camera to IMU seems to fit the translation in the transformation matrix.
Has anyone else looked at this problem earlier or does it exist other calibration files not attached to the dateset?
hi maradol,
Actually, we separately calibrate the extrinsic parameters between the left and right cameras (for ZED2)and the IMU，so when you combine them, there are some obvious accumulated error. Currently, my suggestion is that you can set the extrinsic paramters we provided as initial guess, you can optimize them again. We will also continue to improve it. Thanks
How would you suggest performing extrinsic calibration between camera and IMU without calibration board footage?
Hi @ecbaum ,
You can refer to https://github.com/HKUST-Aerial-Robotics/VINS-Fusion
Assume that you have known the camera intrinsics parameters, then I think you can use the VINS-Fusion framework to online estimate the extrinsics without calibration board.
Hi @maradol ,
Recently, we revised the extrinsic parameters, and evaluate the new extrinsic parameters.
As the figure shows that, the final accumulation error on the x axis is : |0.158-0.12|=0.038 meters. 0.12 meter is baseline of ZED2.
The extrinsic parameters between left and right seems to be reasonable. However we can observe that the translational component of the extrinsics from the cameras to the IMU are around 3.0 and 0.7 meters in the Z and Y directions. Is this really in accordance to the schematics of the setup? Is the origin of the IMU possibly shifted?
Dear @Kailthen, thanks for your interest in our work. Check with your notebook that the extrinsic for the UrbanNav-HK-Medium-Urban-1 seems good but the calibration errors are larger in the 0314 data. You can try to fine-tune the parameter as the LiDAR intensity can be an indicator as shown in the below figures (by matching the door jamb and the car plate).
We might need some time to fine-tune the parameters. We will update once we have a better value.
Test r3live with UrbanNav-HK-Medium-Urban-1:
The image size is 672, 376, may be it's too small to get RGB.
@Kailthen the results of r3live are interesting! we plan to share a higher resolution (720P for stereo and 1080P for mono) of the data this year, please stay tuned!
close as no further update