kujason/avod

Lower average predicion of pedestrian class on KITTI validation split than paper

Closed this issue · 4 comments

bluecdm commented

Hi, I'm trying to train AVOD-FPN in my PC and evaluate on KITTI validation split.
When I used the given 'pyramid_people_example.config' in training and calculated AP after training, I found maximum 3D detection AP at moderate difficulty is 0.478 at step 116000.
However, in Table III of the paper, the 3D pedestrian detection AP on validation split at moderate difficulty is 0.588.

Do I need to modify the config file to achieve higher AP?

My PC setting is
Ubuntu 16.04
python 3.5.6
tensorflow-gpu 1.11.0

Thank you for your great code and explanation.

WuYiYiYi commented

Hi,I have a question that how to run the AP in this program?Can you help me?Thanks a lot!

WuYiYiYi commented

I dont know how to run the kitti-native-eval.When I make the makefile,it will happen error.

mikon1995 commented

hi, @bluecdm have you reproduced the accuracy for cyclist 3D AP in val split? avod shows low robust for people class and the results on kitti test set remains to drop 10-20% from the acc results of paper. i can't solve it till now

  • 👍1
kujason commented

We found the AP results for pedestrians and cyclists to not be very stable, mostly due to the low number of available labels for those two classes in KITTI.