zfhxi/pytorch-ddp-demo

For learning distributed data parallel training in Pytorch

torch分布式训练demo

1. demo01参考https://zhuanlan.zhihu.com/p/113694038，原仓库https://github.com/yangkky/distributed_tutorial/tree/master/src

2. demo02参考https://spell.ml/blog/pytorch-distributed-data-parallel-XvEaABIAAB8Ars0e、https://zhuanlan.zhihu.com/p/98535650，代码参考多个仓库：https://github.com/spellml/deeplab-voc-2012/tree/master/models、https://github.com/tczhangzhi/pytorch-distributed/blob/master/horovod_distributed.py