seungeunrho/minimalRL

Wrong formula for calc-target in SAC?

Opened this issue 5 months ago · 0 comments

BeFranke commented 5 months ago

See https://github.com/seungeunrho/minimalRL/blob/c8efed8481e3cd40e9739cfde220a55522555b57/sac.py#L127C1-L127C54

Shouldn't the formula be target = r + gamma * (1 - done) * (min_q + entropy)?