Could you please also provide an example to pretrain on C4?

[hohoCode]

Thanks for the great work!

Could you please also provide an example to pretrain on C4 ? Once trained, is there a way to convert the trained weights to huggingface format model?

Thanks!