Pointers to distributed training

[posenhuang]

Hi,

I am wondering whether it is possible to have multiple gpu training across nodes? Any pointer would be helpful.
Thanks!

[michaelauli]

You can probably do this with nccl by now but we do not support it in this project.

[posenhuang]

Thanks!