ghimiredhikura/Complex-YOLOv3

cuda OOM. I already reduce the batch-size. I do not know what else i can do

Caroline-6 opened this issue · 0 comments

(xiao)xiaotongchen@amax:~/Complex-YOLOv3-master$ python train.py
/home/xiaotongchen/miniconda2/envs/xiao/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:455: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_qint8 = np.dtype([("qint8", np.int8, 1)])
/home/xiaotongchen/miniconda2/envs/xiao/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:456: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_quint8 = np.dtype([("quint8", np.uint8, 1)])
/home/xiaotongchen/miniconda2/envs/xiao/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:457: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_qint16 = np.dtype([("qint16", np.int16, 1)])
/home/xiaotongchen/miniconda2/envs/xiao/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:458: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_quint16 = np.dtype([("quint16", np.uint16, 1)])
/home/xiaotongchen/miniconda2/envs/xiao/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:459: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_qint32 = np.dtype([("qint32", np.int32, 1)])
/home/xiaotongchen/miniconda2/envs/xiao/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:462: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
np_resource = np.dtype([("resource", np.ubyte, 1)])
Namespace(batch_size=1, epochs=300, evaluation_interval=2, gradient_accumulations=2, img_size=608, model_def='config/complex_yolov3.cfg', multiscale_training=True, n_cpu=8, pretrained_weights=None)
Traceback (most recent call last):
File "train.py", line 38, in
model = Darknet(opt.model_def).to(device)
File "/home/xiaotongchen/miniconda2/envs/xiao/lib/python3.6/site-packages/torch/nn/modules/module.py", line 381, in to
return self._apply(convert)
File "/home/xiaotongchen/miniconda2/envs/xiao/lib/python3.6/site-packages/torch/nn/modules/module.py", line 187, in _apply
module._apply(fn)
File "/home/xiaotongchen/miniconda2/envs/xiao/lib/python3.6/site-packages/torch/nn/modules/module.py", line 187, in _apply
module._apply(fn)
File "/home/xiaotongchen/miniconda2/envs/xiao/lib/python3.6/site-packages/torch/nn/modules/module.py", line 187, in _apply
module._apply(fn)
File "/home/xiaotongchen/miniconda2/envs/xiao/lib/python3.6/site-packages/torch/nn/modules/module.py", line 193, in _apply
param.data = fn(param.data)
File "/home/xiaotongchen/miniconda2/envs/xiao/lib/python3.6/site-packages/torch/nn/modules/module.py", line 379, in convert
return t.to(device, dtype if t.is_floating_point() else None, non_blocking)
RuntimeError: CUDA error: out of memory