Can not load the tokenizer.

[Tianhang-Lyu]

Hello! Dear cell2sentence developers. The peoject is excellent！
However, I encounted an issue, which I can not load the tokenizer while can normally load the model. The screen below show this issue. I really hope your response on this issue and maybe a helpful documentation will help us to learn this package.

[sachalevy]

Hi @Tianhang-Lyu ,

Thanks for your interest in cell2sentence!

It seems like you're passing your model path to the transformers.AutoTokenizer.from_pretrained method. Since we're only fine-tuning the model, we do not edit the tokenizer. You should be able to load it directly by passing the name of the model to the from_pretrained function.

In short, you can swap pretrained_model_name_or_path with the name of the model like "gpt2".

I hope this is helpful! Let me know if you have any other questions.