Slight Error regarding translating radar data between coordinate frames
Closed this issue · 0 comments
I think there is a slight issue with translating between different coordinate frames when constructing the radar data and preparing it for training.
In the get_radar_data()
function inside nuscenesdataset.py
file we have the code excerpt:
# Get reference pose and timestamp.
ref_sd_token = sample_rec['data']['RADAR_FRONT']
ref_sd_rec = nusc.get('sample_data', ref_sd_token)
ref_pose_rec = nusc.get('ego_pose', ref_sd_rec['ego_pose_token'])
ref_cs_rec = nusc.get('calibrated_sensor', ref_sd_rec['calibrated_sensor_token'])
ref_time = 1e-6 * ref_sd_rec['timestamp']
# Homogeneous transformation matrix from global to _current_ ego car frame.
car_from_global = transform_matrix(ref_pose_rec['translation'], Quaternion(ref_pose_rec['rotation']),inverse=True)
The end result is all the radar data in stored in reference to the ego-frame of when RADAR_FRONT was captured for the current record.
Then in train_nuscenes.py
we have:
rad_xyz_cam0 = utils.geom.apply_4x4(cams_T_velo[:,0], xyz_rad)
which translates the radar data from the ego-frame of when RADAR_FRONT was captured into the coordinate system of the REFERENCE CAMERA, as the 0th index represents the transformation from ego-frame to REF_CAMs coordinate frame.
To my understanding, when creating the radar data in get_radar_data()
, shouldn't we instead be originally translating the radar data into ego-frame of when REFERENCE CAMERA was captured if we wanted everything to be perfect? ie:
ref_sd_token = sample_rec['data']['REF_CAM']
The difference in position in ego-frame between these two would not be substantial at all, and I do not think it would provide much benefit but is this correct? Thank you