Is it normal for training to take several days?
Closed this issue · 2 comments
OldAAAA commented
I try to train on the megadepth dataset with the scenes provided by the file in the path "d2-net-master/megadepth_utils/train_scenes.txt",but i find the training time is very long.So i want to know is it normal? if not , how long should be taken on training?
`100%|██████████████████████| 77/77 [01:29<00:00, 1.16s/it]
97%|██████████▎ | 1816/1875 [12:40<00:21, 2.72it/s, loss=1.0919]`
my hardware configure is as follow:
GPU:Gefore GTX 1060ti
Cpu: intel corei5 9th Gen
What kind of hardware configuration is recommended?
mihaidusmanu commented
Hello. 12 minutes per epoch seems reasonable to me. I have only trained on 1080Ti / 2080Ti - from what I remember the training took somewhere in the range 2-5 hours.
OldAAAA commented
OK,Thank you!