Train new language models?
mrmarbury opened this issue · 3 comments
mrmarbury commented
I am wondering if (and how) this can be used to train my own model. I have seen that the huggingface tokenizer has a train function. How can I utilise this function from this library? Sorry for asking this is all pretty new to me.
cigrainger commented
We don't currently support it but we should! I'll look into how we might support it. Thanks for reminding me. :)
mrmarbury commented
awesome.❤️🔥 Let me know if I can help you in some way :)