WongKinYiu/yolov7

Expected accuracy for YOLOv7-tiny leakyRelu

atangfan opened this issue · 3 comments

atangfan commented

What is the expected accuracy for YOLOv7-tiny leakyRelu?
I got a very low AP @ 0.75 by using COCO2017-val

 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=100 ] = 0.458
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=100 ] = 0.007

Should this be the same as SILU version?
Could you share experiments for YOLOv7 with leakyRelu if you have? Thanks!

WongKinYiu commented

YOLOv7-tiny
416: 35.2% AP, 52.8% AP50, 37.3% AP75.
320: 30.8% AP, 47.3% AP50, 32.2% AP75.

python train.py --workers 8 --device 0 --batch-size 64 --data data/coco.yaml --img 512 512 --cfg cfg/training/yolov7-tiny.yaml --weights '' --name yolov7-tiny --hyp data/hyp.scratch.tiny.yaml
chjej202 commented

Hi, @WongKinYiu

what is the coco 2017 val accuracy with yolov7-tiny leaky?

I got the following mAP with your uploaded yolov7-tiny.pt (val2017)
python test.py --data data/coco.yaml --img 416 --batch 32 --conf 0.001 --iou 0.5 --device 0 --weights yolov7-tiny.pt --name yolov7tiny_416_val

Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.331
Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=100 ] = 0.505
Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=100 ] = 0.345
Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.117
Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.359
Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.541
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 1 ] = 0.291
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 10 ] = 0.457
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.489
Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.227
Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.553
Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.724

I also tested with the darknet with your uploaded yolov7-tiny.cfg and yolov7-tiny.weights (val2017)
./darknet detector map ./cfg/coco.data ./cfg/yolov7-tiny.cfg yolov7-tiny.weights
mean average precision (mAP@0.50) = 0.558828, or 55.88 %

I got different results from your yolov7-tiny.weights and yolov7-tiny.pt, and my results are not equal to your accuracy mentioned on the above comment.

what is the expected accuracy should I get?

WongKinYiu commented

The yolov7-tiny.pt is uploaded for someone who want to use it for onnx/tensorrt testing. That is the reason why yolov7-tiny is not included in readme of main branch.

We develop tiny models on darknet compatible branch, you could use yolor or pytorch yolov4 to get expected accuracy.