How long will it take to finish a training process?

[bb0928]

Thanks for your project, works pretty well for me. But now I am wondering a question that how long will it take to complete the whole process of training. I read the code, which indicates that total time steps is 5e6, and in each epochs it takes 5e3 time steps, according to my understanding, it will takes 1000 epochs to finish it.
I have trained it for 12 hours, with ~50 epochs finished, if my understanding above is correct, then it is very time-consuming for me to finish the training. Is there any misunderstanding toward my situation? What's more, if I terminate it now, will it continue training according to the result of this time? or it just train it again as very beginning?
Thanks a lot.

[reiniscimurs]

Hi,

In my experience 100 epochs should be enough to train the model. You can see a deeper discussion here: #91

At each training time the model is initialized with random weights. If you want to continue training the model with saved weights, you need to load them with the load method.

[bb0928]

Thanks for your reply, will keep trying.

[Sau1-Goodman]

Hello, how can I "load them with the load method"? Should I add some parameters after the python3 train_velodyne_td3.py command? Can you help me? Thank you very much!