uvavision/RerankingTransformer

Cannot reproduce the reuslts of alphaQE+RRT

yuyouxixi opened this issue · 1 comments

Hello, i tried to reproduce your results shown in the iccv paper, while cannot get the consistent results.
image
I have checked all the hyperparameters to ensure them to be consistent with those in the paper. Do you have any idea with this?

fwtan commented

Thank you for pointing this out! I confirm that there is an error in Table 3 of our paper: the numbers for QE+RRT were obtained by reranking the top-200, instead of top-100 of the images. These can be verified by changing this line of code. I'm sorry for this mistake. The results do not change the observation made in the paper: combining QE and RRT outperforms QE-only and RRT-only. I will submit a revised version of the paper to arxiv.