KITTI result on Yolov3

I have trained KITTI dataset around 20k iter with yolov3 to reproduce your result in paper and archive mAP like this with your setting in paper. Through training process, i saw loss value was converging but i don't know why mAP for Car at easy is quite low. Did you see it normal? Can you give me some ideas about this result?

@CuongNguyen218
Hi, if you train the model using the half of the KITTI training set, make sure you have a lot of car class in your training data.

Hi @jwchoi384,

I have listed the number of labels for each class and it seems that the car's box still dominates. I have no idea about this results

@CuongNguyen218
Is your model YOLOv3? or Gaussian YOLOv3?
Please train your model again from the beginning.
It is different each time you run it because weight is initialized randomly.
In addition, we trained the model around 50-60k iteration.

@jwchoi384 ,
Tks you. Firstly, I train my model with yolov3 and gaussian yolov3 in the future to reproduce some results in your paper. Secondly, my initial weight is darknet53.conv.74 and I continue train it in 30k iter. I hope i can generate some results like yours.

@CuongNguyen218
Batch and subdivision sizes also have a impact on performance.
Our batch size is 64 and subdivision is 16, and we use 4 gpus for training.

@jwchoi384 ,
Hi, after 50k iters, i got this result at the end of training process.

But when i use devkit code, the results is so bad for car class

My cfg: batchsize 32 and sub 32 because I can't set batchsize =64 with single GPU

In eval code, threshold which I set is in the image above?

@CuongNguyen218
Hmm... I also evaluated the trained model using devkit code, you mentioned above.
In addition, I didn't modify anything in the evaluation code when calculating the mAP.
As I know, KITTI evaluation method is slight different with COCO and PASCAL VOC.
(KITTI: Detections which are smaller than the minimum size do not count as false positive.)
http://www.cvlibs.net/datasets/kitti/eval_object.php?obj_benchmark=2d
So, I think the mAP result using devkit is right result.

Did you use below detection threshold for evaluating the KITTI dataset using devkit?

Gaussian_YOLOv3/examples/detector.c

Line 537 in b3c0ec2

float thresh = 0.005;

@jwchoi384 ,
maybe i miss thresh =0.005 because I used AlexeyAB repo to train my model. I will try again. Should I have to retrain or just revalid my trained model?

@CuongNguyen218
You just revalid your trained model.
You don't need to retrain your model.

@jwchoi384 ,
.
I used your detector code but i want to switch eval to kitti. How do i do that ?

@CuongNguyen218
You must change the eval = ??? in xxx.data file.

Gaussian_YOLOv3/cfg/KITTI_3cls.data

Line 6 in b3c0ec2

eval = kitti

I don't know why my results is so bad. Loss still converged, mAP increase. But when I use your detector.c to valid my model, I got results like image below. This results are lower than result i get above ?

@CuongNguyen218
I don't know why...
If you use "AlexeyAB" darknet, please check the validation code and detection threshold.