About NuScenes
Opened this issue · 9 comments
Hi, thank you for your good paper with beautiful code. I want to know how to train and test (data split) on the NuScenes dataset, and would you consider releasing the corresponding code?
We have not yet organized the relevant code for NuScenes, but you can refer to the KITTI-360 dataset code for modification. And we use NuScenes-Mini for convenience. The scenario sequence is shown in Appendix Table S6, which represents the index number of the v1.0-mini dataset (merged LIDAR_TOP in the samples and sweeps folders). The data split is the same as KITTI-360 dynamic sequences.
OK, thank you
Hi, I have another question. For the pose loading of nuscenes, should I load the corresponding pose in ego_pose.json? Do I need to make any transformation, such as applying the translation and rotation of calibrated_sensor.json or other files?
Hi.
Right, you can refer to the official utils in nuscenes to obtain the sensor poses. Specifically, you can get the transform_matrix lidar2car from calibrated_sensor and car2world from ego_pose. Then the final pose lidar2world would be car2world@lidar2car.
Thank you for your reply. I have obtained the correct pose.
However, during the training process, I found that except for the Seq 450-500, the loss of all other sequences became NAN after training for a few steps in the first epoch. I haven't changed any parameter settings. Is there anything wrong with my experimental setup?
Hello.
Maybe you should visualize the point cloud sequence first to see if it is correct and continuous.
After that, check the configuration of the nuscenes dataset.
For example, modify the H W intrinsics for range-view generation, and calculate scale offset of this sequence to generate corresponding txt config file.
The following is recommended: H = 32 W = 1024 fov_lidar = [10.0, 40.0].
Thank you very much for your answer. I will continue to try and if there are any questions, I will ask you again.
Hi, sorry to bother you.
I want to confirm with you that the Seq ID I have chosen is correct. Based on the sampling frequency of nuScenes (20HZ) given in the paper, which is consistent with KITTI (10HZ), I selected one frame per frame interval in the sequences.
The corresponding starting and ending Seq IDs are as follows:
Seq 450-500: n008-2018-08-01-15-16-36-0400__LIDAR_TOP__1533151606948845.pcd -> n008-2018-08-01-15-16-36-0400__LIDAR_TOP__1533151611946499.pcd
Seq 1250-1300: n008-2018-08-28-16-43-51-0400__LIDAR_TOP__1535489300196771.pcd -> n008-2018-08-28-16-43-51-0400__LIDAR_TOP__1535489305246972.pcd
Seq 1600-1650: n008-2018-08-30-15-16-55-0400__LIDAR_TOP__1535657110100754.pcd -> n008-2018-08-30-15-16-55-0400__LIDAR_TOP__1535657115097816.pcd
Seq 2200-2250: n015-2018-10-02-10-50-40+0800__LIDAR_TOP__1538448756347324.pcd -> n015-2018-10-02-10-50-40+0800__LIDAR_TOP__1538448761348305.pcd
Seq 3180-3230: n015-2018-11-21-19-38-26+0800__LIDAR_TOP__1542800849297337.pcd -> n015-2018-11-21-19-38-26+0800__LIDAR_TOP__1542800854297767.pcd
According to my visualization, the overall shape is similar to that of the paper. But I'm not sure if the frame ID and sampling strategy I've chosen are correct. Thank you.
Hi, there.
It looks right and you can just have a try. 🙌