/agent

Examples of agents for neodroid environments 💡

Primary LanguagePythonApache License 2.0Apache-2.0

header

Agent

This repository will host all initial machine learning efforts applying the Neodroid platform.

Neodroid is developed with support from Research Council of Norway Grant #262900. (https://www.forskningsradet.no/prosjektbanken/#/project/NFR/262900)

Build Status Coverage Status GitHub Issues GitHub Forks GitHub Stars GitHub License

python opencv pytorch

numpy tqdm matplotlib

Contents Of This Readme

Algorithms

Algorithms Implemented

  1. Deep Q Learning (DQN) (Mnih et al. 2013)
  2. DQN with Fixed Q Targets (Mnih et al. 2013)
  3. Double DQN (DDQN) (Hado van Hasselt et al. 2015)
  4. DDQN with Prioritised Experience Replay (Schaul et al. 2016)
  5. Dueling DDQN (Wang et al. 2016)
  6. REINFORCE (Williams et al. 1992)
  7. Deep Deterministic Policy Gradients (DDPG) (Lillicrap et al. 2016 )
  8. Twin Delayed Deep Deterministic Policy Gradients (TD3) (Fujimoto et al. 2018)
  9. Soft Actor-Critic (SAC & SAC-Discrete) (Haarnoja et al. 2018)
  10. Asynchronous Advantage Actor Critic (A3C) (Mnih et al. 2016)
  11. Syncrhonous Advantage Actor Critic (A2C)
  12. Proximal Policy Optimisation (PPO) (Schulman et al. 2017)
  13. DQN with Hindsight Experience Replay (DQN-HER) (Andrychowicz et al. 2018)
  14. DDPG with Hindsight Experience Replay (DDPG-HER) (Andrychowicz et al. 2018 )
  15. Hierarchical-DQN (h-DQN) (Kulkarni et al. 2016)
  16. Stochastic NNs for Hierarchical Reinforcement Learning (SNN-HRL) (Florensa et al. 2017)
  17. Diversity Is All You Need (DIAYN) (Eyensbach et al. 2018)

Environments Implemented

  1. Bit Flipping Game (as described in Andrychowicz et al. 2018)
  2. Four Rooms Game (as described in Sutton et al. 1998)
  3. Long Corridor Game (as described in Kulkarni et al. 2016)
  4. Ant-{Maze, Push, Fall} (as desribed in Nachum et al. 2018 and their accompanying code)

Requirements

  • pytorch
  • tqdm
  • Pillow
  • numpy
  • matplotlib
  • torchvision
  • torch
  • Neodroid
  • pynput

(Optional)

  • visdom
  • gym

To install these use the command:

pip3 install -r requirements.txt

Usage

Export python path to the repo root so we can use the utilities module

export PYTHONPATH=/path-to-repo/

For training a agent use:

python3 procedures/train_agent.py

For testing a trained agent use:

python3 procedures/test_agent.py

Results

Target Point Estimator

Using Depth, Segmentation And RGB images to estimate the location of target point in an environment.

REINFORCE (PG)

DQN

DDPG

PPO

GA, EVO, IMITATION...

Perfect Information Navigator

Has access to perfect location information about the obstructions and target in the environment, the objective is to navigate to the target with colliding with the obstructions.

REINFORCE (PG)

DQN

DDPG

PPO

GA, EVO, IMITATION...

Contributing

See guidelines for contributing here.

Licensing

This project is licensed under the Apache V2 License. See LICENSE for more information.

Citation

For citation you may use the following bibtex entry:

@misc{neodroid-agent,
  author = {Heider, Christian},
  title = {Neodroid Platform Agents},
  year = {2018},
  publisher = {GitHub},
  journal = {GitHub repository},
  howpublished = {\url{https://github.com/sintefneodroid/agent}},
}

Other Components Of the Neodroid Platform

Authors

Here other contributors to this project are listed.