Finetune for Remi XL
joanroig opened this issue · 2 comments
Hello there, I was wondering if this torch implementation of Remi XL can be finetuned as the original Remi project, I tried generating my own dataset representations and resume the training with it but seems like the dictionary changed and I cannot continue training the original dataset. Using the original dictionary crashes the training and using the new one is not allowed due a change in the model (different n_token value in config.yaml). I tried several times and I was also not able to finetune the CP successfully.
Training from scratch in both implementations is working fine, thanks for it! 👍🏼
I was wondering if this torch implementation of Remi XL can be finetuned as the original Remi project
The answer is no. The REMI presentation in this paper is slightly different. We removed some redundant tokens to make the sequence shorter than the original version. It can be seen as improved REMI representation.