Release reduced testset?

I can help but notice that each scene has a lot of images that are very correlated. This leads to evaluation on my local machine taking multiple hours (about 1 sec for matching, and 1 sec for ransac).

Considering the images in the query sequences are very correlated, simply taking every 10th frame ought to give almost identical results.

I get that I could do this manually for the train/val sets, but I'm simply interested in eval for benchmarking.

This large number of images also is somewhat inconvenient purely in size, downloading the precomputed depth + the testset took over 100GB, which is quite substantial.

Ok, so in the dataloader:

map-free-reloc/lib/datasets/mapfree.py

Line 173 in 1df14c1

sample_factor = {'train': 1, 'val': 5, 'test': 5}[mode]

There already seems to be stride = 5
Is this correct, in that case, when I cache matches, do I only need to compute [::5] ?

So if I change this:

map-free-reloc/etc/feature_matching_baselines/compute.py

Line 80 in 1df14c1

for pair in tqdm(im_pairs_path):

to use [::5], that should produce correct results?

Not quite, also had to change it in the dataloader (duh).

Indeed we sample the test/validation frames by a factor of 5. However, we still compute correspondences between the reference frame and every query image, without the subsampling. Your proposed change in compute.py would address this, but you'd need to make sure that the correspondence loader

map-free-reloc/lib/models/matching/feature_matching.py

Line 41 in 1df14c1

corr = self.correspondences[pair_id]

would load the correspondences for the correct frame.

I believe this could be accomplished by replacing the line

map-free-reloc/lib/datasets/mapfree.py

Line 155 in 1df14c1

'pair_id': index*self.sample_factor,

with
'pair_id': index,

We also provide pre-computed correspondences in case you are using SIFT/SG/LoFTR, but if you are using another method you will need to recompute them.

Thanks :)