evaluate AP is all 0

Question

evaluate AP is all 0

Jung-jongwon opened this issue a year ago · 9 comments

Hi, thank you for your work.
I got all AP is 0 when i train graph_rcnn_voi.
Can you tell me what is the problem??

Car AP@0.70, 0.70, 0.70:
bbox AP:0.0000, 0.0000, 0.0000
bev AP:0.0000, 0.0000, 0.0000
3d AP:0.0000, 0.0000, 0.0000
Car AP_R40@0.70, 0.70, 0.70:
bbox AP:0.0000, 0.0000, 0.0000
bev AP:0.0000, 0.0000, 0.0000
3d AP:0.0000, 0.0000, 0.0000
Car AP@0.70, 0.50, 0.50:
bbox AP:0.0000, 0.0000, 0.0000
bev AP:0.0000, 0.0000, 0.0000
3d AP:0.0000, 0.0000, 0.0000
Car AP_R40@0.70, 0.50, 0.50:
bbox AP:0.0000, 0.0000, 0.0000
bev AP:0.0000, 0.0000, 0.0000
3d AP:0.0000, 0.0000, 0.0000

Answer 1 · 2023-04-05T09:08:53.000Z

Please use the command bash scripts/dist_ts_train.sh to train the graph_rcnn_vo/voi separately. You can directly load the weights of the one-stage model (second_mini+centernet) and then only train the refinement network.

Answer 2 · 2023-05-16T04:19:16.000Z

I also got all AP is 0 when i train graph_rcnn_voi.

Answer 3 · 2023-05-16T05:03:21.000Z

hi, how do you train the model? Can you also provide the scripts?

Answer 4 · 2023-05-16T08:36:09.000Z

I just debug in my computer,and I don't use scripts.A train in the mini-kitti show that the AP is all 0.

Answer 5 · 2023-05-16T08:37:13.000Z

just like him

2023-05-16 16:31:24,218 INFO Start evaluation kitti_models/graph_rcnn_po(default)
2023-05-16 16:31:24,219 INFO Loading KITTI dataset
2023-05-16 16:31:24,220 INFO Total samples for KITTI dataset: 43
2023-05-16 16:31:24,222 INFO ==> Loading parameters from checkpoint /home/hsh/下载/GD-MAE/output/kitti_models/graph_rcnn_po/default/ckpt/checkpoint_epoch_1.pth to GPU
2023-05-16 16:31:24,351 INFO ==> Checkpoint trained from version: pcdet+0.5.1+abd05ce
2023-05-16 16:31:24,380 INFO ==> Done (loaded 301/301)
2023-05-16 16:31:24,391 INFO *************** EPOCH 1 EVALUATION *****************
eval: 100%|██████████| 43/43 [00:07<00:00, 5.50it/s, recall_0.3=(43, 36) / 128]
2023-05-16 16:31:32,210 INFO *************** Performance of EPOCH 1 *****************
2023-05-16 16:31:32,210 INFO Run time per sample: 0.1535 second.
2023-05-16 16:31:32,210 INFO Generate label finished(sec_per_example: 0.1817 second).
2023-05-16 16:31:32,210 INFO recall_roi_0.3: 0.335938
2023-05-16 16:31:32,210 INFO recall_rcnn_0.3: 0.281250
2023-05-16 16:31:32,210 INFO recall_roi_0.5: 0.039062
2023-05-16 16:31:32,211 INFO recall_rcnn_0.5: 0.023438
2023-05-16 16:31:32,211 INFO recall_roi_0.7: 0.000000
2023-05-16 16:31:32,211 INFO recall_rcnn_0.7: 0.000000
2023-05-16 16:31:32,211 INFO Average predicted number of objects(43 samples): 0.000
2023-05-16 16:31:52,293 INFO Car AP@0.70, 0.70, 0.70:
bbox AP:0.0000, 0.0000, 0.0000
bev AP:0.0000, 0.0000, 0.0000
3d AP:0.0000, 0.0000, 0.0000
Car AP_R40@0.70, 0.70, 0.70:
bbox AP:0.0000, 0.0000, 0.0000
bev AP:0.0000, 0.0000, 0.0000
3d AP:0.0000, 0.0000, 0.0000
Car AP@0.70, 0.50, 0.50:
bbox AP:0.0000, 0.0000, 0.0000
bev AP:0.0000, 0.0000, 0.0000
3d AP:0.0000, 0.0000, 0.0000
Car AP_R40@0.70, 0.50, 0.50:
bbox AP:0.0000, 0.0000, 0.0000
bev AP:0.0000, 0.0000, 0.0000
3d AP:0.0000, 0.0000, 0.0000

2023-05-16 16:31:52,293 INFO Result is save to /home/hsh/下载/GD-MAE/output/kitti_models/graph_rcnn_po/default/eval/eval_with_train/epoch_1/val
2023-05-16 16:31:52,293 INFO Evaluation done.*
2023-05-16 16:31:52,298 INFO Epoch 1 has been evaluated
2023-05-16 16:33:22,376 INFO End evaluation kitti_models/graph_rcnn_po(default)

Answer 6 · 2023-05-16T11:45:31.000Z

Hi, I am not sure whether the model could achieve desirable results with only 43 training samples and 1 epoch. Maybe you could try it with more samples and epochs.

Answer 7 · 2023-05-18T07:41:14.000Z

2023-05-18 15:13:53,805 INFO *************** Performance of EPOCH 1 *****************
2023-05-18 15:13:53,806 INFO Run time per sample: 0.1001 second.
2023-05-18 15:13:53,806 INFO Generate label finished(sec_per_example: 0.1035 second).
2023-05-18 15:13:53,806 INFO recall_roi_0.3: 0.000000
2023-05-18 15:13:53,806 INFO recall_rcnn_0.3: 0.000000
2023-05-18 15:13:53,806 INFO recall_roi_0.5: 0.000000
2023-05-18 15:13:53,806 INFO recall_rcnn_0.5: 0.000000
2023-05-18 15:13:53,806 INFO recall_roi_0.7: 0.000000
2023-05-18 15:13:53,806 INFO recall_rcnn_0.7: 0.000000
2023-05-18 15:13:53,816 INFO Average predicted number of objects(3769 samples): 0.000
2023-05-18 15:14:06,675 INFO Car AP@0.70, 0.70, 0.70:
bbox AP:0.0000, 0.0000, 0.0000
bev AP:0.0000, 0.0000, 0.0000
3d AP:0.0000, 0.0000, 0.0000
Car AP_R40@0.70, 0.70, 0.70:
bbox AP:0.0000, 0.0000, 0.0000
bev AP:0.0000, 0.0000, 0.0000
3d AP:0.0000, 0.0000, 0.0000
Car AP@0.70, 0.50, 0.50:
bbox AP:0.0000, 0.0000, 0.0000
bev AP:0.0000, 0.0000, 0.0000
3d AP:0.0000, 0.0000, 0.0000
Car AP_R40@0.70, 0.50, 0.50:
bbox AP:0.0000, 0.0000, 0.0000
bev AP:0.0000, 0.0000, 0.0000
3d AP:0.0000, 0.0000, 0.0000

2023-05-18 15:14:06,683 INFO Result is save to /share/home/105324/GD-MAE/output/kitti_models/graph_rcnn_voi/default/eval/eval_with_train/epoch_1/val
2023-05-18 15:14:06,683 INFO Evaluation done.*
2023-05-18 15:14:06,715 INFO Epoch 1 has been evaluated
2023-05-18 15:15:36,810 INFO End evaluation kitti_models/graph_rcnn_voi(default)

Answer 8 · 2023-05-18T08:00:19.000Z

Maybe even if I train 80 epoch，the sequence could be same.But I don't know why?

Answer 9 · 2023-05-18T11:15:43.000Z

Hi, please refer to issue#12, which may be helpful.