xuelunshen/gim

Understanding the normalization in Lightglue matcher

stepstefan opened this issue · 1 comments

Hi, great work and amazing results!

I was going through Lightglue matching code, trying to understand it's api and performance and noticed reversed image sizes in matcher code:

        size0 = data["resize0"][:, [1, 0]]
        size1 = data["resize1"][:, [1, 0]]
        kpts0 = normalize_keypoints(kpts0, size0).clone()
        kpts1 = normalize_keypoints(kpts1, size1).clone()

This in turn mismatches axis for normalization and causes coordinates greater than 1 as input into positional encoding.
Is this something that was done on purpose and if so, why? If it was a bug, was similar input used during training procedure?

The reason for doing it this way here is to maintain consistency with the operations in training. If I remember correctly, this way was also follow the operations of a version of gluefactory.