CUDA out-of-memory

[smittal10]

Hi,
It is mentioned in the "Transfer Learning Examples" section that you fine-tuned the pre-trained DALL-E on 8 V100 GPUs. I tried running you transfer_learning_ex.py script on V100 GPUs (16GB GPU memory per CPU). It throws CUDA OOM error. Can you please share the exact specs of the hardware you used for this?

[saehoonkim]

Thanks for your interest to this project. We used 8 V100 with 32GB memory for transfer learning examples. If you're using GPUs with limited memory, you may need to consider model parallelization.