cannot reproduce DA results from Table I in the paper
aabramovrepo opened this issue · 2 comments
Hi,
I'm trying to reproduce results of the Domain Adaptive Faster R-CNN for GTA Sim 10k -> Cityscapes domain shift (last row in Table I in your paper). However, I'm getting much lower numbers somehow.
My training data looks as follows (trainval.txt file):
source_3384639
source_3384643
source_3384645
...
target_aachen_000000_000019_leftImg8bit
target_aachen_000001_000019_leftImg8bit
target_aachen_000002_000019_leftImg8bit
...
source_* are GTA Sim 10k images from Driving in the Matrix, while target_* are Cityscapes train images (2975). Therefore, 12975 images altogether in trainval. In GTA Sim 10k annotations I have replaced motorbike
class by Cityscapes' motorcycle
.
The test data (test.txt) consists of Cityscapes val images (500):
target_frankfurt_000001_023369_leftImg8bit
target_frankfurt_000001_075296_leftImg8bit
target_frankfurt_000000_006589_leftImg8bit
...
For the training I'm just following instructions from your GitHub page:
./tools/train_net.py --gpu 0 --solver models/da_faster_rcnn/solver.prototxt --weights data/imagenet_models/VGG16.v2.caffemodel --imdb voc_2007_trainval --iters 70000 --cfg models/da_faster_rcnn/faster_rcnn_end2end.yml
I'm getting the following car AP
values:
10k: 32.72
20k: 31.12
30k: 33.51
40k: 33.98
50k: 30.90
60k: 33.53
70k: 33.94
These numbers are much lower as compared to 38.97
from the paper. Do you have any idea what might be wrong in my pipeline?
Thanks a lot in advance for your help! Very much appreciated!
Best,
Alexey
Hi Alexey, an example of sim10k to cs is provided in Caffe2, you could check it out:
https://github.com/krumo/Detectron-DA-Faster-RCNN
Hello, can you reproduce the results now? I have encountered similar problems. Do you use all the categories of SIM 10k or only one category of car when training the source domain?