GPT-neo 350M weights?

Question

GPT-neo 350M weights?

gangiswag opened this issue 2 years ago · 3 comments

Would it be possible to also share weights for the 350M parameter model, since I don't see it here: https://huggingface.co/EleutherAI

Asking since GPT-neo 350M is comparable with BERT-large (345M) in terms of number of parameters.

Answer 1 · 2022-01-06T14:02:01.000Z

Unfortunately, we don't have a 350M parameter version of GPT-Neo. Would you be able to use a 350M parameter GPT-NeoX model? Unfortunately it's not compatible with the HuggingFace API yet.

Answer 2 · 2022-02-17T21:18:58.000Z

@gangiswag https://huggingface.co/xhyi/PT_GPTNEO350_ATG/tree/main

Answer 3 · 2022-02-18T16:57:50.000Z

@gangiswag https://huggingface.co/xhyi/PT_GPTNEO350_ATG/tree/main

I want to stress that this is not an EleutherAI-endorced model. I have no idea what it's contents or performance are.