Some image pairs show a large number of incorrect matches

[tudipffmgt]

Hi,

I am working on a very difficult dataset and for most of the image pairs your method works very well! Thank you.
But for some other images I get a large number of matches where the majority is visibly incorrect. Clearly, I would prefer that those image pairs show no matches at all.
At the moment I am calculating D2-net keypoints and descriptors for every possible image combination.
Would there be are smarter way or some tweaks to eliminate the incorrect matched pairs?

Edit: I am not using the multiscale feature at the moment

[mihaidusmanu]

Hello. Are those matches before or after Geometric Verification? I would expect most of them to get filtered in that step.

Are you using any match filtering techniques? You can try using ratio test (in the paper we suggested 0.95 for multiscale but you might want to tune that) - this is not perfect since it eliminates a large amount of good matches as well but it can help in some cases.

Lastly, you can try to use a retrieval system (NetVLAD, R-MAC, DIR, ...) and only match images with close retrieval descriptors - this should also filter out some of the completely unrelated image pairs.

[tudipffmgt]

Hello. Are those matches before or after Geometric Verification? I would expect most of them to get filtered in that step. Are you using any match filtering techniques? You can try using ratio test (in the paper we suggested 0.95 for multiscale but you might want to tune that) - this is not perfect since it eliminates a large amount of good matches as well but it can help in some cases.

At the moment I am using the standard routine explained in https://github.com/tsattler/visuallocalizationbenchmark/tree/master/local_feature_evaluation (modify_database_with_custom_features_and_matches). So it is already after Geometric Verification and for most images I get reasonable results. Could I add the ratio test somewhere in the modify_database_with_custom_features_and_matches.py there?

Lastly, you can try to use a retrieval system (NetVLAD, R-MAC, DIR, ...) and only match images with close retrieval descriptors - this should also filter out some of the completely unrelated image pairs.

Thanks, I could try that.

[mihaidusmanu]

That implementation only uses a mutual NN matcher (see L124): https://github.com/tsattler/visuallocalizationbenchmark/blob/773c593a33998681934e5228005a0e2c380e74b5/local_feature_evaluation/modify_database_with_custom_features_and_matches.py#L124

You would have to replace it by a ratio test / ratio test + MNN matcher. One implementation is available here: https://github.com/mihaidusmanu/torch-feature-matching/blob/4bb6c4a8172af4b28834ffb115c666ae021226ef/feature_matchers.py#L31

[tsattler]

@mihaidusmanu Do you also want to include the ratio test matcher into the localization benchmark code?

[mihaidusmanu]

@tsattler I will make a note and add it later today / tomorrow morning :)!