MarvinTeichmann/MultiNet

error in train.py ResourceExhaustedError OOM when allocating tensor with shape [1,256,92,309]

MaybeShewill-CV opened this issue · 4 comments

MaybeShewill-CV commented

I use python train.py --hypes hypes/multinet2.json script to train a multinet2 model on Kitti dataset. I use a single NVIDIA GTX 1070 GPU with 8GB graphics memory and my computer memory is 16GB. Does somebody know whether my computer has not got enough memory to train a multinet2 model on kitti dataset? If so, how much memory does i need to complete this job.Thanks for replying.

villanuevab commented

See #18. In particular, the author's response:

GPU Memory. You need 12 GB to train MultiNet with VGG encoder on full resolution and 16 GB for MultiNet with Resnet encoder. You can use patch training or resize the images to train on lower resolution.

One suggestion is to preprocess the images to reduce their size. If you can share the full stacktrace we might be able to troubleshoot further.

  • 👍2
  • 😄1
MaybeShewill-CV commented

@villanuevab Thanks for replying. I will try your method as soon as possible. Thanks a lot.

yinhai86924 commented

@villanuevab Thanks for replying. I will try your method as soon as possible. Thanks a lot.

跑 multinet2 6G显存 出行OOM batchsize都为1 不知道怎么办了

yinhai86924 commented

I use python train.py --hypes hypes/multinet2.json script to train a multinet2 model on Kitti dataset. I use a single NVIDIA GTX 1070 GPU with 8GB graphics memory and my computer memory is 16GB. Does somebody know whether my computer has not got enough memory to train a multinet2 model on kitti dataset? If so, how much memory does i need to complete this job.Thanks for replying.

跑 multinet2 6G显存 出行OOM batchsize都为1 不知道怎么办了