vstark21/RL_gym

Implementing different reinforcement learning algorithms on different gym environments and comparing results.

RL gym

Implementing different reinforcement learning algorithms on different gym environments.

These algorithms are implemented in this repo:

• A2C
• DDPG
• Double DQN
• Dueling DQN
• TD3

And tested on these environments.

Cartpole	Pendulum	Acrobat	Lunar Lander Continuous

A2C

A2C is a on-policy, model-free reinforcement learning algorithm. Here is the pseudo code for A3C which is almost similar to A2C.

Agent trained using A2C playing Acrobat game.

DDPG

DDPG is a off-policy, model-free reinforcement learning algorithm. Here is the pseudo code for DDPG

Agent trained using DDPG playing lunar lander game.

Double_DQN

Double DQN is a off-policy, model-free reinforcement learning algorithm. Here is the pseudo code for Double DQN

Agent trained using Double DQN playing Cartpole game.

Dueling_DQN

Similar to DDQN, dueling network contains two separate estimators: one for the state value function and one for the state-dependent action advantage function.

Formula for the decomposition of Q-value:

• θ is shared parameter for the network.
• α parameterizes output stream for advantage function Α.
• β parameterizes output stream for value function V.

Agent trained using Dueling DQN playing Acrobat game.

TD3

TD3 is a off-policy, model-free reinforcement learning algorithm. Here is the pseudo code for TD3

Agent trained using TD3 playing Pendulum game.

References

• Reinforcement Learning - Goal Oriented Intelligence
• Reinforcement Learning by Sentdex
• OpenAI's Spinning Up Docs
• Wang et al., Dueling Network Architectures for Deep Reinforcement Learning
• Dueling Deep Q Networks
• Deriving Policy Gradients and Implementing REINFORCE
• Understanding Actor Critic Methods and A2C
• Keras DDPG Example

© V I S H W A S