Hidden States in RecurrentDepthBackbone

[TextZip]

Hi, I was wondering why the hidden_states are not being reset when the episode ends in the learn_vision pipeline from the on_policy_runner. I know that they are being detached but when the robot gets reset shouldn't it also have a fresh hidden state of none instead of the previous episode's hidden_state value ?

Thanks for the amazing repo..

[chengxuxin]

You are right. But we found this does not affect training but saves a lot of effort if you want to find all terminating envs and reset corresponding hidden states.