jaromiru/AI-blog

A3c getsample potential error

ai2010 opened this issue · 1 comments

On your blog page on theory about A3C:
https://jaromiru.com/2017/03/26/lets-make-an-a3c-implementation/
you put define the getsample function in the Agent class:
def get_sample(memory, n):
r = 0.
for i in range(n):
r += memory[i][2] * (GAMMA ** i)
s, a, _, _ = memory[0]
_, _, , s = memory[n-1]

return s, a, r, s_

but in the actual code at line 183 the for loop is missing like:

def get_sample(memory, n):
s, a, _, _ = memory[0]
_, _, , s = memory[n-1]

return s, a, self.R, s_

I think your blog is right the implementation is missing the part right? Thanks

The code in this repo is the more efficient version explained under Effective implementation.