horoiwa/deep_reinforcement_learning_gallery

Bug: r2d2/atari multistep-return

Opened this issue · 0 comments

horoiwa commented

deep_reinforcement_learning_gallery/R2D2/atari/r2d2_main.py

Line 163 in 8a9e329

TQ = rewards + self.gamma * (1 - dones) * next_maxQ # (unroll_len, batch_size)

deep_reinforcement_learning_gallery/R2D2/atari/actor.py

Line 163 in 8a9e329

TQ = rewards + self.gamma * (1 - dones) * next_maxQ # (unroll_len, batch_size)

next_maxQへの割引率はgamma ** n_stepが正しい