OptimalScale/LMFlow

baichuan-inc_Baichuan2-7B-Chat can't training

xiaohangguo opened this issue · 2 comments

感觉是baicihuan2的tokenizer做了更新,可能需要适配一下
报了个这样的错误:
Tokenizer class BaichuanTokenizer does not exist or is not currently imported.

版本的话是用的最新的代码,v0.05。
感觉改一下tokenizer应该就可以。

Thanks for your interest in LMFlow! We will import this feature in the near future. We also encourage any contributions on this matter. If you are interested in contributing to LMFlow, please feel free to add a new pull request 🤗

哈哈,好的,如果我能在你们更新之前pr的话