UCAS-AOD-benchmark

A benchmark of UCAS-AOD dataset. (Only Oriented box is tested)

Introduction

There is no official division of the UCAS-AOD dataset, thus it's troublesome to compare the performance on different models. You can directly make comparison with our test results if you adopt the same division strategy.

Dataset repare

Download UCAS-AOD dataset .
Unzip dataset package into your root_dir, and rename the folder to UCAS_AOD.
Run data_prepare.py (modify the dataset dir to your own), and you will obtain directory as follow:

UCAS_AOD
└───AllImages
│   │   P0001.png
│   │   P0002.png
│   │	...
│   └───P1510.png
└───Annotations
│   │   P0001.txt
│   │   P0002.txt
│   │	...
│   └───P1510.txt       
└───ImageSets   
└───Test
│   │   P0003.png
│   │	...
│   └───P1508.txt 
└───CAR
└───PLANE
└───Neg

Put our imageset files train.txt, val.txt and test.txt into ImageSets folder.
Train, eval and test you model according to ImageSets settings.

notes: The integrated dataset contains 1510 images, with train set 755, val set 302, test set 452(following DOTA division 5:2:3). Files are numbered from 1-1510, in which 1-510 are cars, 511-1510 are airplanes. Besides, classname is attached to label file in format of classname x1 y1 x2 y2 x3 y3 x4 y4 theta cx cy w h ,

for example:

car  2.763971e+02	9.125021e+01	2.911375e+02	3.823406e+01	3.308891e+02	4.928647e+01	3.161486e+02	1.023026e+02	1.055379e+02	2.787673e+02	3.876027e+01	4.975157e+01	6.301615e+01	
car  3.002141e+02	1.003123e+02	3.209637e+02	4.665470e+01	3.566901e+02	6.047021e+01	3.359405e+02	1.141279e+02	1.111416e+02	3.055889e+02	4.856245e+01	4.572642e+01	6.365764e+01	
...

Experiment

Environment

NVIDIA 2080 Ti
pytorch>1.1.0
CUDA 10.0

Details

Models are Trained on trainset , and test on testset, valset is used for parameter optimization.
All models were trained with a batch size of 4, totally 100 epochs, and the trained models are available at Baidu Drive.
lr is set to 0.0001 (Adam), we warm up lr for first 500 iters, and adjust lr with step 70 and 90 epochs. Finally, best weight will be tested and uploaded.
Data augment is adopted (random flip, hsv augment, translation, rotation).
All models are evaluated via VOC07 metric which is different DOTA and COCO.

Benchmark

model	backbone	input_size	car	airplane	mAP	paper link	remarks
RetinaNet	ResNet50	800*800	84.64	90.51	87.57	——	code, 3anchors
Faster RCNN(OBB)	ResNet50	800*800	86.87	89.86	88.36	——	code
RoI Transformer	ResNet50	800*800	87.99	89.90	88.95	CVPR2019	code
RetinaNet+DAL	ResNet50	800*800	89.25	90.49	89.87	——	code

Some Results

Notes : More results and PRs are welcomed if you test with imagesets division here.

wuyunxiangwyx/UCAS-AOD-benchmark