Detector - Mosaic data augmentation
AlexeyAB opened this issue · 27 comments
Mosaic-data augmentation (like CutMix augmentation): #3114 (comment)
-
Mosaic-data mAP increasing: #3114 (comment)
-
comparison MixUp and CutMix: #3272 (comment)
Use:
[net]
mosaic=1
Currently works only if Darknet is compiled with OpenCV.
New mosaic:
Old mosaic=1 - CLICK ME to view
Implemented.
Something goes wrong with
[net]
mosaic=1
chart:
cfg file:
yolov3-spp-giou-gs-mos.cfg.txt
without mosaic chart:
For me, mosaic gave me -0.4% mAP - don't really know if that's just random fluctuation or it did actually hurt performance a little.
I improved mosaic=1 for Detector. Try new mosaic.
Two commits: 87f36b7 and 13f064f
results were updated here #3874 (comment)
@Kyuuki93 It seems that new mosaic=1 significantly improves AP@75
- spp,giou,gs(iou_n=0.50,uc_n=0.50) | - | | 91.39% AP@50 |
58.01% AP@75 - spp,giou,gs,mosaic(n=0.50,uc_n=0.50) | - | | 90.48% AP@50 |
60.02% AP@75
Also it seems that iou_n=0.07 is too low value for GIoU, and iou_n=0.5 is much better.
@Kyuuki93 It seems that new mosaic=1 significantly improves AP@75
- spp,giou,gs(iou_n=0.50,uc_n=0.50) | - | | 91.39% AP@50 |
58.01% AP@75- spp,giou,gs,mosaic(n=0.50,uc_n=0.50) | - | | 90.48% AP@50 |
60.02% AP@75Also it seems that
iou_n=0.07is too low value for GIoU, andiou_n=0.5is much better.
And I updated mosaic results iou_thresh = 0.213, the improvement at AP@75 still exists but not that significant
- spp,giou,gs(in=0.5,un=0.5,it=0.213) |
91.89%AP@50 |63.53%AP@75| - spp,giou,gs,mosaic(in=0.5,un=0.5,it=0.213) |
91.82%AP@50 |63.89%AP@75|
@Kyuuki93 So both iou_thresh=0.213 and mosaic=1 increase accuracy.
Maybe mosaics will give a greater increase in accuracy on large datasets
For me, mosaic gave me -0.4% mAP - don't really know if that's just random fluctuation or it did actually hurt performance a little.
Speaking of training accuracy fluctuation I did an experiment on a small private dataset (see #3874 (comment) for dataset details).
I trained 15 instances of "Yolo v3 Tiny 3L" for 10_000 steps on the same dataset with same config file and parameters.
The randomness of training brings some fluctuation in accuracy (mAP@.5):
- extend (max - min mAP value) is 2.18%
- standard deviation is 0.62% around a mean of 82.18%
Std Dev and extend are higher for classes harder to detect.
What pre-trained weights-file did you use for training yolo_v3_csresnext50-pan-spp.cfg there? #3874 (comment)
I don't use any pre-trained weights in all my experiments, every network is trained from scratch.
@laclouis5 (I don't use any pre-trained weights) - This can reduce accuracy, the larger the model, the more pre-trained weights are needed.
I try to train my networks in a very similar context to obtain a fair comparison. I don't fully control and understand how pre-trained weights affect initialisation and training as pre-trained weights could be different between networks (Tiny and plain Yolo for instance), so I prefer to train from scratch to compare apples to apples.
I also observe that on small database and small network (Yolo v3 Tiny) training curve is steeper but the final accuracy (mAP@.5) is roughly the same as it plateaus rapidly near 3_000 iterations in my case.
In the following image are overlapped training curves with and without pre-trained weights (from a network I previously trained).
Maybe pre-trained weights / transfert learning is more relevant and can increase accuracy for deeper networks such as yolo_v3_csresnext50-pan-spp.cfg and/or with bigger databases with more classes.
That being said, when deploying a network I would certainly train with Coco weights to get the best result.
@AlexeyAB I'll train yolo_v3_csresnext50-pan-spp.cfg with pre-trained weights to quantify the difference on this one.
Maybe pre-trained weights / transfert learning is more relevant and can increase accuracy for deeper networks such as yolo_v3_csresnext50-pan-spp.cfg and/or with bigger databases with more classes.
I think so.
I'll train yolo_v3_csresnext50-pan-spp.cfg with pre-trained weights to quantify the difference on this one.
Yes, try and compare accuracy.
Is ok to use mosaic=1 and mixup=1 at the same time? @AlexeyAB
@tianfengyijiu Yes, but will be used only Mosaic.
@AlexeyAB Here is the result:
| Model | Training Chart | mAP@.5 | mAP@[0.5...0.95] |
|---|---|---|---|
| Yolo v3 Spp Panet |  |
83.38% | 42.21% |
| Yolo v3 Spp Panet with pre-trained weights |  |
88.34% | 45.88% |
The pre-trained version is much more stable, +0.30% mAP@0.5 and +3% mAP@[0.5...0.95] higher than my best non pre-trained network.
If I have time l'll train other networks with pre-trained weights to be able to compare this result against.
Continuing issue with mosaic in Detector experienced here : #4432 (comment)
I'm indeed using the newest repo and the issue seems to only occur when mosaic is used with letter_box and images are positioned in such a way that they overlap awkwardly. All other images come out correctly.
For example :
This issue goes away when turning off letter_box as this prevents images running over one another, it seems. For example :
@BernoGreyling Thanks! Yes, it seems there is an issue for combination letter_box=1 mosaic=1
hi @AlexeyAB
Is this problem solved ? ( combination letter_box=1 mosaic=1 )
Not yet
can you share the paper for mosaic, i want cite, but not find in Google Scholar, THX
can you share the paper for mosaic, i want cite, but not find in Google Scholar, THX
mosaic cames from yolov3 or yolov4
@BernoGreyling Thanks! Yes, it seems there is an issue for combination
letter_box=1 mosaic=1
Hi @AlexeyAB
Has this bug been fixed yet?
Yes,
Just use
[net]
mosaic=1
mosaic_bound=1
letter_box=1
In my experiments, usually this "bug" improves accuracy, so usually it is better to train without mosaic_bound=1, because in the most cases we need to detect the whole object:
mosaic_bound=1- increases accuracy for occluded objects (it detects only visible part of object)mosaic_bound=0- increases accuracy for not-occluded objects (it detects visible+occluded parts of object) - usually it is more important, because we have much more not-occluded objects than occluded
https://github.com/AlexeyAB/darknet/wiki/CFG-Parameters-in-the-%5Bnet%5D-section
mosaic_bound=1 - limits the size of objects when mosaic=1 is used (does not allow bounding boxes to leave the borders of their images when Mosaic-data-augmentation is used)