the-eye.eu is down again, is there a mirror?

[nepeee]

Can't get the pretrained XL model right now :(

[StellaAthena]

You can download it from HuggingFace

[nepeee]

But i can't use the training code with the HuggingFace files and this code is the only one that can train the big models on colab/kaggle notebook TPU-s.

[StellaAthena]

@leogao2 is there an eaidata download link for MTF GPT-Neo?

@nepeee that’s weird… HF isn’t super optimized but it shouldn’t be so inefficient you can’t finetune a 2.7B model on a v3-8 TPU…

[nepeee]

I tried the pytorch TPU example from HF, it got killed do to lack of TPU memory with the 1.3B model.(no model parallelism)
https://github.com/huggingface/transformers/tree/master/examples/pytorch/language-modeling

Also tried the jax example but its not working in notebooks do to google/jax#8300 (works on TPU VMs)
https://github.com/huggingface/transformers/tree/master/examples/research_projects/jax-projects/model_parallel

This code works well both on colab/kaggle.

[StellaAthena]

Okay, I’ll see what I can find. We are currently working with the eye to get then back online and to set up alternative torrents in the event of future outages.

[nepeee]

Torrent can be a good alternative. Thank you for the help!