About training time
Opened this issue · 4 comments
s0mnus112 commented
Hello!
I wanted to ask about your training time,I only ran 50 epochs in 12 hours while training the semi model.
Yanfeng-Zhou commented
Training time is related to many factors, such as GPU, dataset scale, model size, image size, etc. You need provide some detailed experimental settings so that I can judge whether this is reasonable.
s0mnus112 commented
Thank you!
I used 4 RTX 3090(24GB),and i changed the batch_size to 8,It's still training slowly. I think maybe my dataset is too big,It's contains 24000 256*256 images.
Yanfeng-Zhou commented
You are right. For a dataset containing 24,000 256×256 images, it is a normal speed.I suggest reducing the image size during training, such as 256→128, reducing the total epochs, and increasing the batch size.
s0mnus112 commented
Thank you so much!!🤞