rabbityl/lepard

Which recall did you report in the paper?

qsisi opened this issue · 2 comments

qsisi commented

Hello! Thanks for open-sourcing this great work, here I got a question about the registration recall reported in the article.
From the code :

recall1 = success1/len(self.loader['test'].dataset)

It seems that you calculate the pair-level recall instead of the scene-level recall, but in the methods you compared with such as D3Feat or PREDATOR, the metric recall is calculated in scene-level but not in pair-level.

Could you give some hints about it?

@qsisi That's indeed a very good point which I did not noticed previously. Yes the pair-level RR is used in our Repo. I re-evaluated Predator's model using the pair-level RR it got: 91.7/62.5 on 3DMatch/3DLoMatch.( i.e. 1.1% increased in 3DMatch). We will update the paper and clarify this.

All metrics are updated as pair-level. closing this.