nicklashansen/dmcontrol-generalization-benchmark

Confusion about Capacity of ReplayBuffer

Closed this issue · 2 comments

RayYoh commented

Hello authors, thx for your great work.
I have one question about the capacity of the replaybuffer code. But acording to the original DrQ code, I find that they use a hyperparameter to set the cabicity and their default parameter is 100000. Is there any reasons to set the capacity by train_steps.

nicklashansen commented

Hi, thanks for your question and apologies that it took a while to get back to you. Setting the capacity to # steps ensures that all samples are kept during training. If you have the memory for it, that should yield you the best results. If you are constrained on memory you can reduce the capacity at the cost of performance. I hope this clears things up!

RayYoh commented

Hi, thanks for your question and apologies that it took a while to get back to you. Setting the capacity to # steps ensures that all samples are kept during training. If you have the memory for it, that should yield you the best results. If you are constrained on memory you can reduce the capacity at the cost of performance. I hope this clears things up!

Got that, Thank you very much.