How long does it take to train a cifar-10 glow model from scratch?
hehaodele opened this issue · 3 comments
Hi,
I am trying to train the glow model on CIFAR-10.
I only have one gpu.
It seems that it take me one day to train only 50 epochs.
Is that normal?
I notice that in the paper, the shown results of CIFAR-10 is at epoch 1800.
(which means it may take me a month to train the glow model :( !)
I would like to know, whether it is possible for me to reproduce this result with only singe gpu.
Any comments would be appreciated. Thanks very much!
If you download the pretrained CIFAR10 model it contains several log files 'train.txt', 'test.txt' and 'print.txt'. In 'train.txt' it prints 'train_time' during training, at the last epoch it says
train_time": 586687
Assuming this is seconds it corresponds to roughly a week of training, keep in mind they used 8 GPUs. Training on a single GPU would then take 8 weeks, roughly 60 days.
Since you could train 50 epochs in one day, it would take you 3000/50=60 days to complete training. So it does not seem there is any discrepancy between your training time and that reported in their logs.
If you download the pretrained CIFAR10 model it contains several log files 'train.txt', 'test.txt' and 'print.txt'. In 'train.txt' it prints 'train_time' during training, at the last epoch it says
train_time": 586687Assuming this is seconds it corresponds to roughly a week of training, keep in mind they used 8 GPUs. Training on a single GPU would then take 8 weeks, roughly 60 days.
Since you could train 50 epochs in one day, it would take you 3000/50=60 days to complete training. So it does not seem there is any discrepancy between your training time and that reported in their logs.
Where do you get the pretrained models?
Where do you get the pretrained models?
It's on the front page, just try search for "pretrain".
