Hyperparameter “seed”
Closed this issue · 2 comments
Hi!
Thanks for sharing your code! RL it's a super interesting topic in which I'm still new, so my apologies in advance if these questions are too obvious.
When running the program, there is a hyperparameter “seed” (defalt: None). I found that when it is not set, the result of each evaluation will be different. How does the seed parameter affect the results of the evaluation? How should I set it when training and testing on an unseen dataset?
Thanks for the help!
Hi @ZimengTan, thank your for your question. By setting the seed, Numpy and PyTorch will generate the same pseudo-random numbers. Without a seed, every time there is some randomness in the algorithm, the output will be different. During evaluation, the agent(s) may start to look for the landmark(s) from different locations. During training, the neural network is randomly initialised and exploratory actions are also random. All of those things affect the final results.
To ensure your results are precisely reproducible, I'd recommend setting the seed to the same number (any will do, e.g. 42). If you want to have some confidence interval for your results, you need to change the seed every time you retrain/reevaluate (or set the seed to None).
In theory, the seed should have little effect on the quality of the trained model. In practice, it happens that some unlucky seeds take a lot longer for the model to learn or, worse, cause the training to diverge.
I hope this helps, let me know if you have any other question.
I got it! Thanks so much for your reply!👍😊