X-PLUG/mPLUG-Owl

Have you tuned the vision model in sft stage?

xmy0916 opened this issue · 2 comments

xmy0916 commented

I found code here: https://github.com/X-PLUG/mPLUG-Owl/blob/main/mPLUG-Owl2/scripts/finetune.sh#L32

you have freezed the vision backbone, but according to your paper, it seems all the parrams have been trained?
image

MAGAer13 commented

Yes, this is just reimplementation of our model. You can enable it during training and you should implement layer-wise learning rate by yourself.

xmy0916 commented

Yes, this is just reimplementation of our model. You can enable it during training and you should implement layer-wise learning rate by yourself.

Thanks for your great job and answer~