© 2018-2020, Anyscale. All Rights Reserved
Welcome to the Anyscale Academy tutorials on Ray, the system for scaling your applications from a laptop to a cluster.
This README tells you how to set up the tutorials, it provides a quick overview of its contents, and it recommends which tutorials to go through depending on your interests.
Tips:
- Anyscale is developing a free, hosted version of these tutorials. Contact us for more information.
- This is an early release of these tutorials. Please report any issues:
- GitHub issues
- The #tutorial channel on the Ray Slack
- If you are attending a live tutorial event, please follow the setup instructions provided well in advance.
- For troubleshooting help, see the Troubleshooting, Tips, and Tricks notebook.
Read one of the following setup sections, as appropriate, then jump to Launching the Tutorials.
There is nothing you need to setup, as the hosted environment will provide everything.
However, consider cloning or downloading a release of the tutorial notebooks and supporting software from the Academy repo, so you have a local copy of everything. The README
provides instruction for local setup, if desired.
Tip: Make sure you download the notebooks you modified during the session to save those changes.
Note: Ray support for Windows is experimental. See these release notes for details. Alternatively, Contact Anyscale for a free hosted option for these tutorials.
Follow these instructions to use the tutorials. Note that some commands can take a while to finish.
Clone the Academy GitHub repo or download the latest release.
Now install the dependencies using either Anaconda or pip
in your Python environment. We recommend using Anaconda.
Python 3.7 is recommended. While Ray supports Python 3.6, a user reported a problem using locales. Specifically, the following code throws an error:
import locale
locale.setlocale(locale.LC_ALL, locale.getlocale())
This tutorial doesn't use locales specifically, but you may run into problems with your default locale.
While Ray supports Python 3.8, some dependencies used in RLlib
(the Ray reinforcement library) are not yet supported for 3.8, at the time of this writing.
If you need to install Anaconda, follow the instructions here. If you already have Anaconda installed, consider running conda upgrade --all
.
Run the following commands in the root directory of this project. First, use conda
to install the other dependencies, including Ray. Then activate the newly-created environment, named anyscale-academy
. Finally, run the provided tools/fix-jupyter.sh
script to install a graphing library extension in Jupyter Lab and perform other tasks.
conda env create -f environment.yml
conda activate anyscale-academy
tools/fix-jupyter.sh
If you are using Windows, see the Fixing Jupyter Lab on Windows section below for an alternative to using tools/fix-jupyter.sh
.
Note that Python 3.7 is used.
You can delete the environment later with the following command:
conda env remove --name anyscale-academy
If you don't use Anaconda, you'll have to install these prerequisites first:
- Python 3.7:
- See notes above about problems with 3.6 and 3.8. Don't use 3.8, but 3.6 may work for you.
- The version of Python that comes with your operating system is probably too old. Try
python --version
to see what you have. - Installation instructions are at python.org.
- Pip: A recent version - consider upgrading if it's not the latest version.
- Installation instructions are at pip.pypa.io.
- Node.js: Required for some of the Jupyter Lab graphics extensions we use.
- Installation instructions are here.
Next, run the following commands in the root directory of this project to complete the setup. First, run the pip
command to install the rest of the libraries required for these tutorials, including Ray. Then, run the provided script to install a graphing library extension in Jupyter Lab and perform other tasks.
pip install -r requirements.txt
tools/fix-jupyter.sh
If you are using Windows, see the Fixing Jupyter Lab on Windows section below for an alternative to using tools/fix-jupyter.sh
.
The tools/fix-jupyter.sh
shell script runs the following commands. If you are using Windows, run them yourself as shown here.
First, see if the following pyviz
extension is installed:
jupyter labextension check --installed "@pyviz/jupyterlab_pyviz"
If not, run this command:
jupyter labextension install "@pyviz/jupyterlab_pyviz"
Finally, run these commands:
jupyter labextension update --all
jupyter lab build
jupyter labextension list
The tutorials will start a local Ray "cluster" (one node) on your machine. When you are finished with the tutorials, run the following command to shut down Ray:
ray stop
Also, when you have finished working through the tutorials, run the script tools/cleanup.sh
, which prints temporary files, checkpoints, etc. that were created during the lessons. You might want to remove these as they can add up to 100s of MBs.
If you decide to delete all the files and directories listed, the following bash
command will do it:
tools/cleanup.sh | while read x; do rm -rf $x; done
Note: A Windows version of this script is TBD.
The previous steps installed Jupyter Lab, the notebook-based environment we'll use for all the lessons. To start run the following command in the project root directory:
jupyter lab
It should automatically open a browser window with the lab environment, but if not, the console output will show the URL you should use.
Tip: If you get an error that
jupyter
can't be found and you are using the Anaconda setup, make sure you activated theanyscale-academy
environment, as shown above.
Here is a recommended reading list, based on your interests:
You Are... | Best Tutorials |
---|---|
A developer who is new to Ray | First, Ray Crash Course, then Advanced Ray |
A developer who is experienced with Ray | Advanced Ray (alpha release) |
A developer or data scientist interested in Reinforcement Learning | Ray RLlib |
A developer or data scientist interested in Hyperparameter Tuning | Ray Tune |
A developer or data scientist interested in accelerated model training with PyTorch | See the Ray SGD lesson in the Ray Tune tutorial |
A developer or data scientist interested in model serving | Ray Serve |
A DevOps engineer interested in managing Ray clusters | Ray Cluster Launcher (forthcoming) |
See the Overview notebook for detailed, up-to-date descriptions for each tutorial and the lessons it contains.
See the Troubleshooting, Tips, and Tricks notebook.
For details on the Ray API and the ML libraries, see the Ray Docs.