qcraftai/simtrack

metric jitter on nuscene valset

dwy927 opened this issue · 4 comments

dwy927 commented

The metirc result is not deterministic.

cmd:
python tools/val_nusc_tracking.py examples/point_pillars/configs/nusc_all_pp_centernet_tracking.py --checkpoint model_zoo/simtrack_pillar.pth --work_dir word_dirs/baseline

For fast, I just use nuscenes v1.0-mini set to test.
It returns the different metric results when I run the above cmd.

image
image
image

chenxuluo commented

The model randomly drops some points during the voxelization step. We plan to release the dynamic voxelization version so the results will be deterministic then.

dwy927 commented

Looking forward to it.

liangchunlan commented 
python ./tools/val_nusc_tracking.py examples/point_pillars/configs/nusc_all_pp_centernet_tracking.py --checkpoint ./model_zoo/simtrack_pillar.pth  --work_dir ./save_dir/v1.0-mini/ --local_rank 0

I got the result. What could be the problem.

image

image

zzm-hl commented

The metirc result is not deterministic.

cmd: python tools/val_nusc_tracking.py examples/point_pillars/configs/nusc_all_pp_centernet_tracking.py --checkpoint model_zoo/simtrack_pillar.pth --work_dir word_dirs/baseline

For fast, I just use nuscenes v1.0-mini set to test. It returns the different metric results when I run the above cmd.

image image image

I want to know how to run the train and test code ,the data preparation is done