albertomontesg/keras-model-zoo

Large batch size problem during training/fine-tuning

Closed this issue · 2 comments

bryanyzhu commented

Hi @albertomontesg, I have the same problem like Pool 2D optimization #4377, which can't train C3D with large batch size. Thanks for your suggestions going with the theano module theano.sandbox.cuda.nnet.

However, I directly search theano.sandbox.cuda.nnet on google, but no useful documentation shows up. I don't which one are your refer to, theano.sandbox.cuda.fftconv.conv3d_fft? or theano.sandbox.cuda.dnn.GpuDnnConv3d, or others? Sorry that I am not familiar with Theano.

Could you advise through(like give me an example about 3d conv, 3d pool), or update the model.py using theano.sandbox.cuda.nnet? Thank you very much.

albertomontesg commented

Hi @bryanyzhu, I suggest you to check it out the fork I have of Keras. I modified the conv3d operation and the 3d poolig. Using my fork I solved GPU memory issues and I could use a larger batch size.
Hope this might be helpfull!

bryanyzhu commented

Hi @albertomontesg Thanks a lot. I tried your fork and it works like a charm.