/udacity_p1_navigation

Primary LanguagePython

Project 1: Navigation

Introduction

In this project, a DQN agent is implemented to navigate (and collect bananas) in a square planar world.

Trained Agent

A reward of +1 is provided for collecting a yellow banana, and a reward of -1 is provided for collecting a blue banana. Thus, the goal of your agent is to collect as many yellow bananas as possible while avoiding blue bananas.

The state space has 37 dimensions and contains the agent's velocity, along with ray-based perception of objects around agent's forward direction. Given this information, the agent has to learn how to best select actions. Four discrete actions are available, corresponding to:

  • 0 - move forward.
  • 1 - move backward.
  • 2 - turn left.
  • 3 - turn right.

The task is episodic, and in order to solve the environment, the agent must get an average score of +13 over 100 consecutive episodes.

Getting Started

  1. Follow the instructions in the course GitHub repository to set up the Python environment. These instructions can be found in README.md at the root of the repository. By following these instructions, you will install PyTorch, the ML-Agents toolkit, and a few more Python packages required to complete the project.

  2. Download the environment from one of the links below. You need only select the environment that matches your operating system:

    (For Windows users) Check out this link if you need help with determining if your computer is running a 32-bit version or 64-bit version of the Windows operating system.

    (For AWS) If you'd like to train the agent on AWS (and have not enabled a virtual screen), then please use this link to obtain the environment.

  3. Place the file in the course GitHub repository, in the p1_navigation/ folder, and unzip (or decompress) the file.

  4. Change the file_name parameter in navigation.py to match the location of the Unity environment that you downloaded.

  • Mac: "path/to/VisualBanana.app"
  • Windows (x86): "path/to/VisualBanana_Windows_x86/Banana.exe"
  • Windows (x86_64): "path/to/VisualBanana_Windows_x86_64/Banana.exe"
  • Linux (x86): "path/to/VisualBanana_Linux/Banana.x86"
  • Linux (x86_64): "path/to/VisualBanana_Linux/Banana.x86_64"
  • Linux (x86, headless): "path/to/VisualBanana_Linux_NoVis/Banana.x86"
  • Linux (x86_64, headless): "path/to/VisualBanana_Linux_NoVis/Banana.x86_64"

For instance, if you are using a Mac, then you downloaded VisualBanana.app. If this file is in the same folder as the notebook, then the line in navigation.py should appear as follows:

env = UnityEnvironment(file_name="VisualBanana.app")

How to run

Traning and demonstration of the DQN agent is implemented in navigation.py. To see the training and watch the result of a smart agent, simply run:\

python navigation.py

navigation.py is using VScode's support for Jupyter-like cells, it can be run like a Jupyter Notebook using VScode's interactive window.