About sincos_pos_embed
qiy20 opened this issue · 5 comments
(https://github.com/hustvl/MIMDet/blob/main/models/benchmarking.py#L592)
I don't understand why do we need to do this? Because self.pos_embed has been initialized by resized pos-emded from pretrain checkpoint.
Hi, @qiy20! We have tried to use learnable resized position embed, it gets about 0.6 bbox AP and 0.4 segm AP improvements. Code and result will be updated after more experiments are done. Thanks for your attention!
@vealocia Thanks for your reply! I want to know which type of pos_embed is used in the reproduction process to get 48.0AP(25e)
@vealocia Thanks for your reply! I want to know which type of pos_embed is used in the reproduction process to get 48.0AP(25e)
48.0 AP is achieved with sincos_pos_embed=True. So it means a re-initialized frozen sin/cos position embedding.
Many thanks!
I believe the issue at hand was addressed, as such I'm closing this. Feel free to ask if you have further questions.