sfujim/TD7

intuition about the encoder, fixed encoder and fixed encoder target

Opened this issue · 0 comments

Hi,
Thanks for this awesome work. I'm curious why it need three encoders, listed encoder, fixed encoder and fixed encoder target. And I find the update of the parameter is encoder -> fixed_encoder, fixed_encoder -> fixed_encoder_target.

To my understand, we may just need 2 such as encoder and encoder target, which is similar to actor/critic and actor_target/critic_target.

Could you please give me some hints?

Thanks a lot!