xinge008/Cylinder3D

CUDA out of memory??

Closed this issue · 2 comments

caozhe1011 commented

I have a Teala T4 GPU, when I run "sh train.sh", it comes an error:
File "/home/anaconda3/envs/By-DJR/lib/python3.7/site-packages/torch/autograd/init.py", line 132, in backward
allow_unreachable=True) # allow_unreachable flag
RuntimeError: CUDA out of memory. Tried to allocate 844.00 MiB (GPU 0; 14.76 GiB total capacity; 9.70 GiB already allocated; 498.75 MiB free; 10.15 GiB reserved in total by PyTorch)
0%| | 1/9565 [00:19<50:33:46, 19.03s/it]

How can I solve this?

xinge008 commented

How about your batch size? Is it still OOM if batchsize=1?

Our experiments are conducted on V100 (32G) with batchsize=2.

caozhe1011 commented

How about your batch size? Is it still OOM if batchsize=1?

Our experiments are conducted on V100 (32G) with batchsize=2.

Thanks, it runs successfully.