vicgalle/gpt-j-api

Alternative to Google TPU VM?

birkenbaum opened this issue · 3 comments

birkenbaum commented

Hello,

I would like to run a local instance of GPT-J, but avoid using Google.

I have little to no experience in machine learning and its requirements, are there other solutions I could use? (What are the requirements for a machine in order to run GPT-J?)

Thank you very much!

vicgalle commented

Hi @birkenbaum

You could use a GPU from AWS, I have not tested it but jax (the library this project is built with) can run in GPU,
though you will probably need to adjust with the device parameters in the server code.

The GPU would need around 16-32 GB of RAM, but you should also take into account is not gonna be as fast as the TPU version due to its optimizations.

birkenbaum commented

Thank you for your quick reply!

That's good to know, thank you. I've researched a bit and saw that there's also Floydhub (closing down August 20) and Paperspace, which seems to offer GPUs with 16GB+ RAM that can be billed by the second.

I'm not at home this week, so I will probably try it later next week and report back and welcome any other suggestions you might have :)

Thank you!

  • 👍1
Ontopic commented

Any luck?