yuantn/MI-AOD

low performance about RetinaNet on PASCAL VOC compared to paper, especially the first cycle.

Closed this issue · 2 comments

trhao commented

Thanks for the amazing work, I wanted to ask some questions. As shown in the figure, my environment and related settings and parameters are completely consistent with the paper, the reproduction results on RetinaNet on Pascal VOC are lower than those in the paper, especially the first cycle based on the 5% labeled dataset, the results are uncertain each experiment (11.5%, 12.4%, 29.9%, 37.5%), question one, the reason for the obvious difference in the results of the first cycle, whether it is because of randomly selected data, or other factors, question two, the reason why the final result cannot reach the results in the paper, the data randomly selected at the beginning? learning rate? batch size, etc. Looking forward to your reply, thanks very much.
result

Thanks for your attention to our work. The answers to your questions are as follows.

Answer for question 1. Theoretically, the reason for the difference in the first cycle is mainly the randomly selected data. But from the comparison of the results of the paper with your results on other GPUs, it seems that the type of GPU also affects the results.

Answer for question 2. Maybe Active Learning itself has a large variation range between trials. As you can see in your experiments, MI-AOD-2 has also outperformed the performance of my paper in the middle cycles, as long as you set the same learning rate and batch size.

trhao commented

Thanks for your answer.