evaluation score

[coper16]

Hi, thank you for your great work!
I finished SuperPoint training by your code.
Thank you very much.

I understand that your code is based on "https://github.com/rpautrat/SuperPoint".
And so I would like to ask 3 question.

I tried to use your evaluation code and pretrain model(by MagicLeap).
I use H-Patches-v(only view point change) evaluation dataset.

and so, why is this homography estimation score(threshold 3 pix) lower than the score of pretrain model by evaluation code of https://github.com/rpautrat/SuperPoint?

I think H-Patches-all(view point change and illumination change) score may have same trend.

Is there difference between your training procedure and https://github.com/rpautrat/SuperPoint?
for example, the number of steps of Homographic Adaption procedure, MS-COCO data augmentation…
I would like to know the precautions when changing from using code of https://github.com/rpautrat/SuperPoint to using your code.

What is difference between SuperPointNet_gauss2 model and SuperPointNet model?

Best,

[eric-yyjau]

Hi @coper16 ,

Thank you for your questions.

1. I think the superpoint repo updates their algorithms and results in 2020.
   When I was working on the project in 2019, the results are similar.
   https://github.com/rpautrat/SuperPoint/tree/13c678d08dbd5b1ebec50fae2dc706262baf47f3

The threshold of NN matching, NMS radius, and number of keypoints may affect the results.
Besides, I think they have changed some training details.
Please let me know if you see the differences.

SuperPointNet is an experiment using groupNorm.
SuperPointNet_gauss2 is the latest model with some additional functions for keypoint extraction and matching. These are useful in my deepFEPE paper.

Thanks.

[coper16]

Thank you for your reply.

I see.
There is one point that I want to know about question(1).

In 2019, could you get same result by pretrain model(by magicleap) score(H-Patches viewpoint change score) of https://github.com/rpautrat/SuperPoint/tree/13c678d08dbd5b1ebec50fae2dc706262baf47f3 ?
My score is 0.60.

[eric-yyjau]

Hi @coper16 ,

I didn't have the evaluation for viewpoint and illumination separately.
The results I had was 0.77 under threshold 3 for the whole H-Patches dataset.

Thanks.

[coper16]

Sorry for the late reply…
I found the difference point between 2019 and 2020 of https://github.com/rpautrat/SuperPoint.
The details of the bug were written in rpautrat/SuperPoint#117.
There may be this bug in your code.
This is just a report.
Thanks.