Error applying gradient for some algorithms

[kmakeev]

OS - Ubuntu 19.4

print(tfp.version)
0.9.0-dev20191113
print(tf.version)
2.1.0-dev20191111

In pg, ac a2c algorithms, I get the error:

in converted code:
/home/konstantin/IdeaProjects/RLs/Algorithms/tf2algos/a2c.py:158 train *
self.optimizer_actor.apply_gradients(
/home/konstantin/anaconda3/envs/tsfl2/lib/python3.7/site-packages/tensorflow_core/python/keras/optimizer_v2/optimizer_v2.py:434 apply_gradients
self._create_slots(var_list)
/home/konstantin/anaconda3/envs/tsfl2/lib/python3.7/site-packages/tensorflow_core/python/keras/optimizer_v2/adam.py:149 _create_slots
self.add_slot(var, 'm')
/home/konstantin/anaconda3/envs/tsfl2/lib/python3.7/site-packages/tensorflow_core/python/keras/optimizer_v2/optimizer_v2.py:574 add_slot
var_key = _var_key(var)
/home/konstantin/anaconda3/envs/tsfl2/lib/python3.7/site-packages/tensorflow_core/python/keras/optimizer_v2/optimizer_v2.py:1065 _var_key
if var._in_graph_mode:

AttributeError: 'ListWrapper' object has no attribute '_in_graph_mode'

But with the algorithms dqn, ddqn, dddqn, dpg... everything works.

After writing this code, the error disappears:

self.optimizer_actor.apply_gradients(
zip(actor_grads, self.actor_net.trainable_variables + self.log_std)
)

[StepNeverStop]

Hi,
I haven't run those algorithms in python3.7, tf 2.1.0 and on ubuntu 19.04, not sure if it is related to these running environments.

BTW, could you tell me the training environment you used, so that I can see if I can reproduce the problem.

And also, I am not sure if you have used my latest submitted code.

Now in ac, a2c, I have written this

self.optimizer_actor.apply_gradients(
                zip(actor_grads, self.actor_net.trainable_variables + [self.log_std])
            )

It looks very similar to how you solved the problem, you can try again, or submit a PR.
thx.

[kmakeev]

OK!. I will create a new branch, make changes to it and send you a Pull Request tonight.
I used the BipedalWalker-v2 Box2D Gym environment

[kmakeev]

PG created.

[StepNeverStop]

Thank you very much for you PR!
Feel free to reopen this issue if still any problem about this!