How to finetune with the pretrained model？

[Dicardo]

Thanks for your amazing job !

I am trying to load the pretrained model and finetune it with the training code from dalle2-pytorch on my own data，but I had some problems that bothered me.

1. After the model loads, I don't run the training code, but directly use the function to save the model weights，just want to verify that the whole process works, But when I replaced original decoder model with my saved model for forward inference, the resulting image became random noise.

2. From what I know, dalle2-laion is trained using dalle2-pytorch code, why can't I reproduce this process, is there any step I am missing?

[Dicardo]

Solved this issue after a time. Considering that this project seems not to be updated any more, I will write down the solution if anyone encounters the same problem

[Elapsedf]

Solved this issue after a time. Considering that this project seems not to be updated any more, I will write down the solution if anyone encounters the same problem↳

Hello, Could you share your solution please? Thanks so much! I also encounter this question