Pytorch_MultiGPU
Single-GPU
python 1_trainer_SingleGPU.pyMulti-GPU (DP)
python 2_trainer_DP.pyMulti-GPU (DDP)
python -m torch.distributed.launch --nproc_per_node 4 3_trainer_DDP.pypython 1_trainer_SingleGPU.pypython 2_trainer_DP.pypython -m torch.distributed.launch --nproc_per_node 4 3_trainer_DDP.py