After reducing the size of the large image to the image size (752 * 480) of the Euroc dataset, the output result became worse

Question

After reducing the size of the large image to the image size (752 * 480) of the Euroc dataset, the output result became worse

SerpentWindy opened this issue 8 months ago · 4 comments

My image size is 1164 * 874, and the image includes the front hood of the car. During the one kilometer journey, VINS did not experience a system reboot, but the trajectory distance and angle were inaccurate. Then I resized and cropped the images, and ran VINS again using the modified images as the dataset. However, there was a "big translation" system reboot error during the run. I want to know why the result is worse after resizing and cutting ?
The reason why I resized and collapsed the image is as follows:

I think the front hatch will affect the effectiveness of feature tracking and visual modeling, so I tried to crop the area of the front hatch.
I think the feature detection parameters I use are all parameters corresponding to the Euroc dataset, and my image size is too large compared to (752,480). These feature detection parameters are not suitable for my image, so I want to resize the image with a size of 1164 * 874 to (752,480).

In summary, I resized and collapsed the 1164 * 874 image to obtain a 752 * 480 size image, with the front hatch removed. Because I changed the image size, I needed to modify the camera intrinsic matrix. Without changing other parameters, I encountered a "big translation" system reboot error while running VINS,

My modification process is as follows:

1.   I resized the image with dimensions of 1164 * 874 to (752,600),
2.   crop the (752,600) size image to (752,480) size,
3.   a new bag was created using  (752,480) images and IMU data,
4.   modify the camera's intrinsic matrix based on my resizing and cropping.

My original fx=910.0, fy=910.0, cx=582.0, cy=437.0,
After resize, fx=910.0752/1164=587.90378, fy=910.0600/874=624.71395881, cx=582.0752/1164=376.0, cy=437.0600/874=300,
After crop, fx=587.90378, fy=624.71395881, cx=376.0, cy=300-(600-480)/2=240.0

Answer 1 · 2024-04-29T14:06:17.000Z

This is my original image

Answer 2 · 2024-04-29T14:07:03.000Z

This is my modified image

Answer 3 · 2024-04-29T14:07:14.000Z

This is my original parameter table：

#camera calibration
model_type: PINHOLE
camera_name: camera

image_width: 1164
image_height: 874
distortion_parameters:
k1: 0.1322964747
k2: -0.2047703774
p1: 0.0000790454
p2: 0.00001595080
projection_parameters:
fx: 9.1e+02
fy: 9.1e+02
cx: 5.82e+02
cy: 4.37e+02

#Extrinsic parameter between IMU and Camera.
estimate_extrinsic: 1 #0 Have an accurate extrinsic parameters. We will trust the following imu^R_cam, imu^T_cam, don't change it.
1 Have an initial guess about extrinsic parameters. We will optimize around your initial guess.
2 Don't know anything about extrinsic parameters. You don't need to give R,T. We will try to calibrate it. Do some rotation movement at beginning.
#If you choose 0 or 1, you should write down the following matrix.
#Rotation from camera frame to imu frame, imu^R_cam
extrinsicRotation: !!opencv-matrix
rows: 3
cols: 3
dt: d
data: [0.0017453, -0.0017423, 0.9999970,
0.9999970, 0.0017484, -0.0017423,
-0.0017453, 0.9999970, 0.0017453]
#Translation from camera frame to imu frame, imu^T_cam
extrinsicTranslation: !!opencv-matrix
rows: 3
cols: 1
dt: d
data: [0.01,0.01,0.01]

#feature traker paprameters
max_cnt: 150 # max feature number in feature tracking
min_dist: 30 # min distance between two features
freq: 10 # frequence (Hz) of publish tracking result. At least 10Hz for good estimation. If set 0, the frequence will be same as raw image
F_threshold: 1.0 # ransac threshold (pixel)
show_track: 1 # publish tracking image as topic
equalize: 1 # if image is too dark or light, trun on equalize to find enough features
fisheye: 0 # if using fisheye, trun on it. A circle mask will be loaded to remove edge noisy points

#optimization parameters
max_solver_time: 0.04 # max solver itration time (ms), to guarantee real time
max_num_iterations: 8 # max solver itrations, to guarantee real time
keyframe_parallax: 10.0 # keyframe selection threshold (pixel)

#imu parameters The more accurate parameters you provide, the better performance
acc_n: 0.08 # accelerometer measurement noise standard deviation. #0.2 0.04
gyr_n: 0.004 # gyroscope measurement noise standard deviation. #0.05 0.004
acc_w: 0.00004 # accelerometer bias random work noise standard deviation. #0.02
gyr_w: 2.0e-6 # gyroscope bias random work noise standard deviation. #4.0e-5
g_norm: 9.81007 # gravity magnitude

#loop closure parameters
loop_closure: 1 # start loop closure
load_previous_pose_graph: 0 # load and reuse previous pose graph; load from 'pose_graph_save_path'
fast_relocalization: 0 # useful in real-time and large project
pose_graph_save_path: "/home/nvidia/vio_output/pose_graph/" # save and load path

#unsynchronization parameters
estimate_td: 0 # online estimate time offset between camera and imu
td: 0.0 # initial value of time offset. unit: s. readed image clock + td = real image clock (IMU clock)

#rolling shutter parameters
rolling_shutter: 1 # 0: global shutter camera, 1: rolling shutter camera
rolling_shutter_tr: 0.029 # unit: s. rolling shutter read out time per frame (from data sheet).

#visualization parameters
save_image: 1 # save image in pose graph for visualization prupose; you can close this function by setting 0
visualize_imu_forward: 0 # output imu forward propogation to achieve low latency and high frequence results
visualize_camera_size: 0.4 # size of camera marker in RVIZ

Answer 4 · 2024-04-29T14:08:22.000Z

This is my modified parameter table：

#camera calibration
model_type: PINHOLE
camera_name: camera

image_width: 752
image_height: 480
#distortion_parameters:
k1: 0.1322964747
k2: -0.2047703774
p1: 0.0000790454
p2: 0.00001595080
#projection_parameters:
fx: 587.90378
fy: 624.71395881
cx: 376.0
cy: 240.0

#Extrinsic parameter between IMU and Camera.
estimate_extrinsic: 1 #0 Have an accurate extrinsic parameters. We will trust the following imu^R_cam, imu^T_cam, don't change it.
1 Have an initial guess about extrinsic parameters. We will optimize around your initial guess.
2 Don't know anything about extrinsic parameters. You don't need to give R,T. We will try to calibrate it. Do some rotation movement at beginning.
#If you choose 0 or 1, you should write down the following matrix.
#Rotation from camera frame to imu frame, imu^R_cam
extrinsicRotation: !!opencv-matrix
rows: 3
cols: 3
dt: d
data: [0.0017453, -0.0017423, 0.9999970,
0.9999970, 0.0017484, -0.0017423,
-0.0017453, 0.9999970, 0.0017453]
#Translation from camera frame to imu frame, imu^T_cam
extrinsicTranslation: !!opencv-matrix
rows: 3
cols: 1
dt: d
data: [0.01,0.01,0.01]

#feature traker paprameters
max_cnt: 150 # max feature number in feature tracking
min_dist: 30 # min distance between two features
freq: 10 # frequence (Hz) of publish tracking result. At least 10Hz for good estimation. If set 0, the frequence will be same as raw image
F_threshold: 1.0 # ransac threshold (pixel)
show_track: 1 # publish tracking image as topic
equalize: 1 # if image is too dark or light, trun on equalize to find enough features
fisheye: 0 # if using fisheye, trun on it. A circle mask will be loaded to remove edge noisy points

#optimization parameters
max_solver_time: 0.04 # max solver itration time (ms), to guarantee real time
max_num_iterations: 8 # max solver itrations, to guarantee real time
keyframe_parallax: 10.0 # keyframe selection threshold (pixel)

#imu parameters The more accurate parameters you provide, the better performance
acc_n: 0.08 # accelerometer measurement noise standard deviation. #0.2 0.04
gyr_n: 0.004 # gyroscope measurement noise standard deviation. #0.05 0.004
acc_w: 0.00004 # accelerometer bias random work noise standard deviation. #0.02
gyr_w: 2.0e-6 # gyroscope bias random work noise standard deviation. #4.0e-5
g_norm: 9.81007 # gravity magnitude

#loop closure parameters
loop_closure: 1 # start loop closure
load_previous_pose_graph: 0 # load and reuse previous pose graph; load from 'pose_graph_save_path'
fast_relocalization: 0 # useful in real-time and large project
pose_graph_save_path: "/home/nvidia/vio_output/pose_graph/" # save and load path

#unsynchronization parameters
estimate_td: 0 # online estimate time offset between camera and imu
td: 0.0 # initial value of time offset. unit: s. readed image clock + td = real image clock (IMU clock)

#rolling shutter parameters
rolling_shutter: 1 # 0: global shutter camera, 1: rolling shutter camera
rolling_shutter_tr: 0.029 # unit: s. rolling shutter read out time per frame (from data sheet).

#visualization parameters
save_image: 1 # save image in pose graph for visualization prupose; you can close this function by setting 0
visualize_imu_forward: 0 # output imu forward propogation to achieve low latency and high frequence results
visualize_camera_size: 0.4 # size of camera marker in RVIZ