This file describes how to use Sonic with Ray and RLLib. We include instructions on how to get the training running on EC2.
Start a p2.8xlarge with the Deep Learning AMI (Ubuntu). In us-west-2, this is ami-d2c759aa.
Activate the TensorFlow environment with
source activate tensorflow_p36
Now install Ray and the RLlib requirements using
pip install ray opencv-python
Next we need to install the gym retro environment. Run
git clone --recursive git@github.com:openai/retro.git gym-retro
cd gym-retro
pip install -e .
Now clone this repo and install it:
cd ~
git clone git@github.com:openai/sonic-on-ray.git
cd sonic-on-ray
pip install -e .
You can then run the training with
cd ~/sonic-on-ray
python train_ppo.py
First install Ray on your laptop with
pip install ray
Now clone the sonic-on-ray repo with
git clone git@github.com:openai/sonic-on-ray.git
And start a cluster with
ray create_or_update sonic-autoscaler.yaml
After the cluster has been started, you will see a message like this:
Started Ray on this node. You can add additional nodes to the cluster by calling
ray start --redis-address 172.31.58.176:6379
from the node you wish to add. You can connect a driver to the cluster from Python by running
import ray
ray.init(redis_address="172.31.58.176:6379")
[...]
To login to the cluster, run:
ssh -i ~/.ssh/ray-autoscaler_us-east-1.pem ubuntu@54.152.27.84
You can now start the hyperparameter search by sshing into the cluster, running
source activate tensorflow_p36
and replacing the ray.init()
call in ~/sonic-on-ray/train_ppo_grid_search.py
by the one printed above and then running the script.