Question about the description of datasets in paper and the annotations in the repo
Opened this issue · 3 comments
Hi, thanks for making this benchmark!
I have a question about the description of datasets in the paper and the annotations in the repo.
In the paper, you say
A. Drone-vs-Bird Detection Challenge
61 videos (85,904 images) for training and 16 videos (18,856 images) for validation
B. Anti-UAV
60 videos (149,478 images) for training and 40 videos (37,016 images) for validation
But in this repo
The dataset A is split by 60 videos (76,817 images) for training and 16 videos (18,856 images) for validation.
The dataset B is split by 80 videos (149,478 images) for training and 20 videos (37,016 images) for validation.
====================================================================
The dataset A should contain the video (2019_10_16_C0003_3633_inspire).
The number of images in dataset B is matched with the paper.
So, I think the description of dataset B in the paper is just a typo.
Am I right?
Ih guyleaf, thanks for your comments.
I have double checked the paper and the repository and the only typo I see is that the Anti-UAV dataset says, in this repository, that there are 140 videos for the validation dataset, when it should be 40, yeah there's a typo in there. I'll change it now.
But regarding the other numbers, I don't see any other mistake. The number of images for the three datasets seems all alright. Do you spot anything else?
Hi, thanks for fast response.
Sorry for describing my question not very clear.
Due I cannot reproduce the annotation json file by the conversion script (because it doesn't set random seed),
I wrote a script to analyze each annotation json file in two datasets to collect the videos which they use.
Then, I found the annotation json file is not really matched with the description of datasets in README.md or in the paper.
You could reproduce the problem by this script.
Drone-vs-Bird Detection Challenge
python detection\utils\list_coco_subset.py datasets\drone-vs-bird\train.json --pattern "_\d+$"
Total number of videos(subsets): 60 <--- here should be 61 videos (I don't see 2019_10_16_C0003_3633_inspire in the annotation file)
76,817 images (tqdm) <--- here should be 85,904 images
python detection\utils\list_coco_subset.py datasets\drone-vs-bird\val.json --pattern "_\d+$"
Total number of videos(subsets): 16
18,856 images (tqdm)
Anti-UAV
python detection\utils\list_coco_subset.py datasets\anti-uav\train-full.json --pattern "_(RGB|IR)_\d+$"
Total number of videos(subsets): 80 <--- here 60?
149,478 images (tqdm)
python detection\utils\list_coco_subset.py datasets\anti-uav\val-full.json --pattern "_(RGB|IR)_\d+$"
Total number of videos(subsets): 20 <--- here 40?
37,016 images (tqdm)
I think the description of Anti-UAV dataset is just a typo because the total number of images is matched.
But in the Drone-vs-Bird Detection Challenge dataset, it seems that the training annotation file doesn't contain 2019_10_16_C0003_3633_inspire video.
Oh I see!! All right, let me run some tests and check it again but it seems that you are right. I'll double check later today and confirm to you later on. I'll also commend this in the readme. Thanks!!