CUDA out of memory. Tried to allocate 20.00 MiB
ForgetThatNight opened this issue · 4 comments
Traceback (most recent call last):
File "train.py", line 164, in
main()
File "train.py", line 160, in main
train_net(args)
File "train.py", line 67, in train_net
logger=logger)
File "train.py", line 111, in train
y_pred = model(x)
File "/home/ml/anaconda3/envs/Tacotron2-Mandarin/lib/python3.6/site-packages/torch/nn/modules/module.py", line 541, in call
result = self.forward(*input, **kwargs)
File "/home/ml/Tacotron2-Mandarin/models/models.py", line 503, in forward
encoder_outputs, mels, memory_lengths=text_lengths)
File "/home/ml/anaconda3/envs/Tacotron2-Mandarin/lib/python3.6/site-packages/torch/nn/modules/module.py", line 541, in call
result = self.forward(*input, **kwargs)
File "/home/ml/Tacotron2-Mandarin/models/models.py", line 404, in forward
decoder_input)
File "/home/ml/Tacotron2-Mandarin/models/models.py", line 360, in decode
attention_weights_cat, self.mask)
File "/home/ml/anaconda3/envs/Tacotron2-Mandarin/lib/python3.6/site-packages/torch/nn/modules/module.py", line 541, in call
result = self.forward(*input, **kwargs)
File "/home/ml/Tacotron2-Mandarin/models/models.py", line 78, in forward
attention_hidden_state, processed_memory, attention_weights_cat)
File "/home/ml/Tacotron2-Mandarin/models/models.py", line 61, in get_alignment_energies
processed_query + processed_attention_weights + processed_memory))
RuntimeError: CUDA out of memory. Tried to allocate 20.00 MiB (GPU 0; 7.79 GiB total capacity; 6.85 GiB already allocated; 13.56 MiB free; 23.24 MiB cached)
2080super上报OOM了,请问怎么处理
你好,我也遇到了这个问题,你解决了么
你好,我也遇到了这个问题,你解决了么
还没有,代码运行在服务器,没办法进行debug调试,应该是计算loss的代码哪里有问题
我在P100上测试通过。你们用的GPU是多大显存?把batch-size减半试试看?在utils.py文件中改。
P100
我在P100上测试通过。你们用的GPU是多大显存?把batch-size减半试试看?在utils.py文件中改。
谢谢回答,是可以通过的, 看来 2080super 训练效果不会太好