does it work without finetuning directly using pretrained model for detection?
lucasjinreal opened this issue · 4 comments
does it work without finetuning directly using pretrained model for detection?
The task layer for object detection (i.e., Mask R-CNN & FPN) is not pre-trained and therefore randomly initialized, so I don't think it will work if you don't update the parameter of the task layer.
Similarly, if you don't allow the 1000-cls classifier to be trained, I believe the MAE representation cannot achieve a reasonable top-1 on ImageNet-1k val set.
@Yuxin-CV thanks for your reply. What if frozen MIM backbone and just finetune the task layer and new classifier? Just wonder how good it can be just direcly using pretrained weights without any modification.
@Yuxin-CV thanks for your reply. What if frozen MIM backbone and just finetune the task layer and new classifier? Just wonder how good it can be just direcly using pretrained weights without any modification.
We didn't try that, but to my knowledge， the mim represention isn't very good at linear probing in imagenet 1k，so I guess the same thing will happen on COCO if you freeze the pretrained represention.
We will try this in the future.
Thank u. I am just out of curiosity, since mim is good at general representation from classification to object detection.