What is the use for "Debug Camera offset"?
Closed this issue · 4 comments
Thanks for your great work!
I find that in the dataparser file:
#####################
# Debug Camera offset
There is a piece of code called "Debug Camera offset", which changes the normal pose reading and rotates it by a certain angle.
Will this affect the results? What is its usage?
And should I comment it out?
def get_camera_poses_tracking(poses_velo_w_tracking, tracking_calibration, selected_frames, scene_no=None):
exp = False
camera_poses = []
opengl2kitti = np.array([[1, 0, 0, 0], [0, -1, 0, 0], [0, 0, -1, 0], [0, 0, 0, 1]])
start_frame = selected_frames[0]
end_frame = selected_frames[1]
#####################
# Debug Camera offset
if scene_no == 2:
yaw = np.deg2rad(0.7) ## Affects camera rig roll: High --> counterclockwise
pitch = np.deg2rad(-0.5) ## Affects camera rig yaw: High --> Turn Right
# pitch = np.deg2rad(-0.97)
roll = np.deg2rad(0.9) ## Affects camera rig pitch: High --> up
# roll = np.deg2rad(1.2)
elif scene_no == 1:
if exp:
yaw = np.deg2rad(0.3) ## Affects camera rig roll: High --> counterclockwise
pitch = np.deg2rad(-0.6) ## Affects camera rig yaw: High --> Turn Right
# pitch = np.deg2rad(-0.97)
roll = np.deg2rad(0.75) ## Affects camera rig pitch: High --> up
# roll = np.deg2rad(1.2)
else:
yaw = np.deg2rad(0.5) ## Affects camera rig roll: High --> counterclockwise
pitch = np.deg2rad(-0.5) ## Affects camera rig yaw: High --> Turn Right
roll = np.deg2rad(0.75) ## Affects camera rig pitch: High --> up
else:
yaw = np.deg2rad(0.05)
pitch = np.deg2rad(-0.75)
# pitch = np.deg2rad(-0.97)
roll = np.deg2rad(1.05)
# roll = np.deg2rad(1.2)
cam_debug = np.eye(4)
cam_debug[:3, :3] = get_rotation(roll, pitch, yaw)
Tr_cam2camrect = tracking_calibration["Tr_cam2camrect"]
Tr_cam2camrect = np.matmul(Tr_cam2camrect, cam_debug)
Tr_camrect2cam = invert_transformation(Tr_cam2camrect[:3, :3], Tr_cam2camrect[:3, 3])
Tr_velo2cam = tracking_calibration["Tr_velo2cam"]
Tr_cam2velo = invert_transformation(Tr_velo2cam[:3, :3], Tr_velo2cam[:3, 3])
camera_poses_imu = []
for cam in camera_ls:
Tr_camrect2cam_i = tracking_calibration["Tr_camrect2cam0" + str(cam)]
Tr_cam_i2camrect = invert_transformation(Tr_camrect2cam_i[:3, :3], Tr_camrect2cam_i[:3, 3])
# transform camera axis from kitti to opengl for nerf:
cam_i_camrect = np.matmul(Tr_cam_i2camrect, opengl2kitti)
cam_i_cam0 = np.matmul(Tr_camrect2cam, cam_i_camrect)
cam_i_velo = np.matmul(Tr_cam2velo, cam_i_cam0)
cam_i_w = np.matmul(poses_velo_w_tracking, cam_i_velo)
camera_poses_imu.append(cam_i_w)
for i, cam in enumerate(camera_ls):
for frame_no in range(start_frame, end_frame + 1):
camera_poses.append(camera_poses_imu[i][frame_no])
return np.array(camera_poses)
I take it a try. After I delete this part, the reconstruction quality of dynamic car will apparently decrease. (I try it in scene06)
Is there any relation?
(I just want to reconstruction the scene using the original poses)
Thanks a lot for your answer!
@xBeho1der @wuzirui
Here is the result after deleting this part:(Training for 1w5 Epoch)
Origin:
this is from the original nsg code. My guess is that the authors manually calibrated the noisy camera poses for better performance.
this is from the original nsg code. My guess is that the authors manually calibrated the noisy camera poses for better performance.
Thanks a lot
Also same file , (the highlighted text) how you calculating this
mars/mars/data/mars_kitti_dataparser.py
Line 1010 in 69b9bf9