A question about DQN_With_Fixed_Q_Targets.

[LLYYKK]

According to the paper, the target network should be updated several steps after local network update, but your code seem to be not like this. In your code, the local network updates are followed by soft updates to the target network. I think there needs to be some time between local network updates and target network updates.

def learn(self, experiences=None):
"""Runs a learning iteration for the Q network"""
super(DQN_With_Fixed_Q_Targets, self).learn(experiences=experiences)
self.soft_update_of_target_network(self.q_network_local, self.q_network_target, self.hyperparameters["tau"])