lisa-lab/pylearn2

extension MonitorBasedSaveBest leaks memory

burgerdev opened this issue · 0 comments

The deepcopy part of the train extension MonitorBasedSaveBest has memory leaks: Running an example script I observe ever growing resident set size. Commenting out this line solved the issue, so I guess deepcopy'ing the model has side effects.

RSS growth:

  • without MonitorBasedSaveBest approx. 10kB/epoch
  • with MonitorBasedSaveBest approx. 500kB/epoch

Edit: added workstation specs.

#!/bin/bash
uname -a
pacman -Q python2-theano
pacman -Q python2
pacman -Q cuda
cd ${HOME}/dev/pylearn2 && git rev-parse HEAD
Linux meepo 4.2.5-1-ARCH #1 SMP PREEMPT Tue Oct 27 08:13:28 CET 2015 x86_64 GNU/Linux
python2-theano 0.7.0-2
python2 2.7.10-2
cuda 7.5.18-1
fe9beee9b922c97bd8e5dd2ac023e1def229da31