USTC-Video-Understanding/I3D_Finetune

std::bad_alloc – massive 60GB memory leak?

shadowlift opened this issue · 1 comments

Hi – I'm trying to finetune with two classes.
Even running the while loop for a single iteration causes the program to take >60GB of memory, which causes

terminate called after throwing an instance of 'std::bad_alloc'
what(): std::bad_alloc
/var/spool/PBS/mom_priv/jobs/1224593.pbs.SC: line 7: 13498 Aborted (core dumped) python finetune.py ucf101 rgb 1

I'm running python2 with sonnet v1.23, the frame size for my jpeg sequences are 224 and they consist of 200 frames. Any ideas on how to fix?

@vra @AlexHu123 @Rhythmblue thanks for your otherwise very helpful work :)