rwightman/gen-efficientnet-pytorch

Memory required for training EfficientNet b0-l2

danielamassiceti opened this issue · 1 comments

danielamassiceti commented

Hi there

Is there a recommended amount of GPU memory needed to train EfficientNets b0 to l2? Currently have another implementation of EfficientNet-b8 but running into memory issues on a 24GB GPU. Will I have the same issues with this implementation?

TIA

rwightman commented

@danielamassiceti yes, you will have problems ... they have significant memory requirements and the validation batch size is already down to 8-4 imgs/GPU for B8, L2 without taking up space for gradients in train. B5, maybe B6 is likely the highest you'd want to go without using clusters of V100 GPUs.

If you're using a memory efficient version of Swish activation in the other version they will likely be the same as here, otherwise you'll maybe see a 15-20% drop in mem usage. Best way to see the limits is try.

  • 👍1