Download / Unzip prebuilt MatathonEnvs into the env folder
pip installs
# ml-agents
cd ml-agents
pip install -e .
# gym-unity
cd gym-unity
pip install -e .
# baselines - does not need to be installed
# stable_baselines
cd stable_baselines
pip install -e .
Status-Hopper
Win10
MacOS
Notes
ml-agents-ppo
score=435(23min)
baselines-ppo2 multiagents
score=943(7min)
score=860(11min)
16 agents, nsteps=128
baselines-ppo2 multiagents non-normalized
score=774(7min)
score=450(11min)
16 agents, nsteps=128
baselines-ppo2 MPIx4
score=594 (42min)
score=583 (82min)
Having problems with mpi + ml-agents on windows. Save is broken for normalized agents
baselines-ppo2 single agent
score=328 (31min)
need to check if 1m steps with mpi == 1m steps with single agent as not clear why it would be faster. Save is broken for normalized agents
baselines-ppo2 MPIx4 TfRunningMeanStd
TfRunningMeanStd fixes save / load but trains slower
baselines-ppo2 single agent TfRunningMeanStd
score=95 (40m)
score=107 (49min)
TfRunningMeanStd fixes save / load but trains slower
baselines-ppo2 MPIx4 non-normalized
score=50 (79min)
(should try training for more steps)
Status-Walker
Win10
MacOS
Notes
ml-agents-ppo
baselines-ppo2 multiagents
score=1371(8min)
score=1439(12min)
baselines-ppo2 multiagents non-normalized
score=1005(12min)
16 agents, nsteps=128
OpenAI.Baselines
Example command lines
To enable Tensorboard
# MacOS:
export OPENAI_LOG_FORMAT='stdout,log,csv,tensorboard'
export OPENAI_LOGDIR=summaries
# Win10:
set OPENAI_LOG_FORMAT=stdout,log,csv,tensorboard
set OPENAI_LOGDIR=summaries
Note: Stable Baselines is a fork of OpenAI.Baselines which addresses some issues with OpenAI.Baselines (main one for me is that OpenAI.Baselines cannot save enviroments with normalized observations)