extension MonitorBasedSaveBest leaks memory
burgerdev opened this issue · 0 comments
burgerdev commented
The deepcopy
part of the train extension MonitorBasedSaveBest
has memory leaks: Running an example script I observe ever growing resident set size. Commenting out this line solved the issue, so I guess deepcopy'ing the model has side effects.
RSS growth:
- without
MonitorBasedSaveBest
approx. 10kB/epoch - with
MonitorBasedSaveBest
approx. 500kB/epoch
Edit: added workstation specs.
#!/bin/bash
uname -a
pacman -Q python2-theano
pacman -Q python2
pacman -Q cuda
cd ${HOME}/dev/pylearn2 && git rev-parse HEAD
Linux meepo 4.2.5-1-ARCH #1 SMP PREEMPT Tue Oct 27 08:13:28 CET 2015 x86_64 GNU/Linux
python2-theano 0.7.0-2
python2 2.7.10-2
cuda 7.5.18-1
fe9beee9b922c97bd8e5dd2ac023e1def229da31