Does your code support multi-gpu training?

[yushuinanrong]

Hi Okan,

Thanks for sharing your code!
I'm trying to run your code on multiple GPUs but it seems the code/pytorch only uses one of my 8 GPUs. I tried to set cuda environment variable 'CUDA_VISIBLE_DEVICES=gpu_ids python mycode', but with no luck. The output of torch.cuda.current_device() is always '0', also 'nvidia-smi' concurs that only GPU 0 was used.

Do you have any idea of how to run your codebase on multiple GPUs?

Edit: It turns out the problem is that the 'batch_size' is too large so that the CPU memory is not enough.