gsarti/t5-flax-gcp

Will you make a fine-tuning guide?

swcrazyfan opened this issue · 2 comments

swcrazyfan commented

I really love what you've done! Your training guide is really great, but I'm trying to fine-tune the XL/3B model. Do you plan to explain how to fine-tune?

Thanks!

gsarti commented

Hi and thanks for the kind words!

For sure I would like to make a guide for the fine-tuning part, too, but I'm currently out of time to work on it. The general idea is to use the run_summarization_flax.py example script from the HF Transformers library. The readme should be a good starting point.

P.S. I will soon release a repo under the name gsarti/it5 which will contain different pointers for my experiments on the Italian T5 models I pretrained, and it will include the version of the aforementioned script I used for my fine-tuning experiments.

Hope it helps!

gsarti commented

@swcrazyfan the repository gsarti/it5 is out now, you can have a look!