EleutherAI/gpt-neox

continue training from a checkpoint with different number of gpu/node

mackmake opened this issue · 1 comments

hi
i get a checkpoint of a model trained on multi node from my friend. can i run it in another node with different number of gpus? how about multi-node with different number of gpus from my friend?

This is a non-trivial feature and hasn't yet been added.

It's still in-progress at #836