Peformance of Resnet18_512.pth

[Cc19245]

Hi, @gmberton

Thanks for your great work!

Now I'm testing the code with resnet18_512.pth. I use small/test/database and small/test/queries_v1 dataset you provided. I use the following command to run the code:

python3 eval.py --backbone ResNet18 --fc_output_dim 512 --resume_model ckpts/resnet18_512.pth --test_set_folder /mnt/H/Dataset/CosPlace

And I got the result like this:

2023-10-24 10:26:56   < CosPlace - #q: 1000; #db: 27191 >: R@1: 67.7, R@5: 77.8, R@10: 81.8, R@20: 85.0

But in the README.md of this repo, the result figure show that R@1 should be more than 90. So what's the problem?

[gmberton]

Hi @Cc19245 , thank you for your interest!

As written in the README.md, those results are on Pitts250k. You can see a full list of results on plenty of datasets in our paper.

The small version of SF-XL is not one of the dataset we test on, because that dataset has been created only as a toy example to debug code, and not as a real test set.

[Cc19245]

Hi @Cc19245 , thank you for your interest!

As written in the README.md, those results are on Pitts250k. You can see a full list of results on plenty of datasets in our paper.

The small version of SF-XL is not one of the dataset we test on, because that dataset has been created only as a toy example to debug code, and not as a real test set.

Oh, I see. Thanks for your quick reply!