/ac-ppo

Actor-Critic and openAI clipped PPO in gym cartpole-v0 and pendulum-v0 environment

Primary LanguagePython

ac-ppo

pytorch implementation Actor-Critic and openAI clipped PPO in gym cartpole-v0 and pendulum-v0 environment

introduction

implement A2C and PPO in pytorch

requirement

  • tensorflow (for tensorboard logging)
  • pytorch (>=1.0, 1.0.1 used in my experiment)
  • gym

a2c

a2c in cartpole and pendulum, the training result shows below

a2c-cartpole a2c.py result of a2c in cartpole-v0

a2c-pendulum a2c_pen.py result of a2c in pendulum-v0, it's quite hard for a2c converge in pendulum..

ppo

ppo-pendulum PPO.py result of ppo in pendulum-v0, somehow still hard to converge..don't know why, any one helps?

ppo improved

ppo-modified PPO_advantage.py more efficient update with generalized advantage estimator (GAE)