shaonannan/Deep-Reinforcement-Learning-CS285-Pytorch

Solutions of assignments of Deep Reinforcement Learning course presented by the University of California, Berkeley (CS285) in Pytorch framework

CS285

Pytorch Version of homework assignments of Deep Reinforcement Learning Course
Presented by Dr. Sergey Levin at University of California, Berkeley

Report Bug

Table of Contents

• About the Project
  • Main Goals
  • Completed So Far
• Getting Started
  • Prerequisites
• Usage
• Roadmap
• Contributing
• License
• Contact

About The Project

In this project, we aim to create a Pytorch version of CS285 course whose Tensorflow 1 version is already available at here.

Main Goals

1. Converting all the Tensorflow 1 code to the newest version of Pytorch
2. The current version Mujoco environment that has been used in this project is old which requires only using Python < 3.6 version. Therefore, we seek to make this project compatible with the newer version of this library and, consequently, Python >= 3.6.

Completed So Far

2. Homework 1, 2, 3, and 4 Tensorflow codes have been fully replaced by Pytorch.

Getting Started

Currently, this project is under development, and the same libraries that have been employed in the Tensorflow version of these assignments plus Pytorch are required for running the assignments of this project. However, we are eager to use the versions of these libraries that are presented in the prerequisites section for the future release of this project.

Prerequisites

The libraries that we want to use in the future are as follows.

• Python >= 3.6
• Gym >= 0.17
• Mujoco-py >= 2.0
• Pytorch >= 1.5.1
• TensorboardX
• Matplotlib
• Ipython
• Moviepy
• OpenCV
• Box2d-py

Usage

The instructions for execution of all of these assignments are given in the Readme documents that are located in each of the homework directories.

Roadmap

See the open issues for a list of known issues.

Contributing

Unfortunately, the current version of this repository is not compatible with the latest versions of libraries, such as Tensorflow and Mojocu-py. As a result, installing the proper versions of these libraries, which can enable you to contribute to this repo, could be a hard challenge. However, since I have been faced with this problem before, I designed a certain number of steps that you can take to install the right versions of these libraries.

1. Create a new Conda environment based on Python 3.5 and install matplotlib, ipython, and pytorch. Then, activate it.

conda create -n cs285_env python=3.5 matplotlib ipython pytorch=1.5.0
source activate cs285_env

2. Clone this repository
3. Install mujoco-py
   1. Get mujoco license key file from its website
   2. Create a .mujoco folder in the home directory and copy the given mjpro150 directory and your license key into it

   mkdir ~/.mujoco/
   cd <location_of_your_license_key>
   cp mjkey.txt ~/.mujoco/
   cd <this_repo>/mujoco
   cp -r mjpro150 ~/.mujoco/

   3. Add the following line to bottom of your .bashrc file:

   export LD_LIBRARY_PATH=~/.mujoco/mjpro150/bin/

   4. Build and install mujoco-py 1.50.1.1. It can be downloaded from this link.

   tar -xzf mujoco-py-1.50.1.1.tar.gz 
   cd mujoco-py-1.50.1.1
   python setup.py install

4. Install rest of the libraries given in contribution_requirements.txt file using pip

pip install --user --requirement contribution_requirements.txt

4. At last, it should be considered that before executing scripts of each homework folder (e.g., hw1), you should allow your code to be able to see 'cs285' by executing the following lines:

cd <path_to_hw>
pip install -e .

License

Distributed under the MIT License. See LICENSE file for more information.

Contact

Erfan Miahi - @erfan_mhi - miahi@ualberta.com

Project Link: https://github.com/erfanMhi/Deep-Reinforcement-Learning-CS285-Pytorch