EMI: Exploration with Mutual Information

In ICML 2019

Hyoungseok Kim^{* 1 2}, Jaekyeom Kim^{* 1 2}, Yeonwoo Jeong^{1 2}, Sergey Levine³, Hyun Oh Song^{1 2}

^{*: Equal contribution, 1: Seoul National University, Department of Computer Science and Engineering, 2: Neural Processing Research Center, 3: UC Berkeley, Department of Electrical Engineering and Computer Sciences}

This codebase contains the source code for our paper, EMI: Exploration with Mutual Information.

Citing this work

Please cite if you find our work helpful to your research:

@inproceedings{kimICML19,
  Author    = {Hyoungseok Kim and Jaekyeom Kim and Yeonwoo Jeong and Sergey Levine and Hyun Oh Song},
  Title     = {EMI: Exploration with Mutual Information},
  Booktitle = {International Conference on Machine Learning (ICML)},
  Year      = {2019}}

Environment setup

Prerequisites

A non-virtual machine with the following components:

Ubuntu 16.04
CUDA 8.0
cuDNN 6.0
Conda

Setting up Conda environment

Run conda env create -f environment.yml.
After activating the created environment by executing conda activate rllab3, run pip install --upgrade https://github.com/Lasagne/Lasagne/archive/master.zip.

Setting up MuJoCo

Create a subdirectory, ./vendor/mujoco/.
Obtain a MuJoCo license for your machine by following the instructions from their website if you don't have one. They offer a number of licensing options including 30-day free trials.
Copy mjkey.txt, the license key file, into ./vendor/mujoco/.
Get the version 1.31 of the MuJoCo binaries for Linux from their website. Unzip the file.
Copy all the files inside the directory mjpro131/bin/ from the extracted content, into ./vendor/mujoco/.

Running experiments

Before running experiments, activate the conda environment by running conda activate rllab3.
To train an EMI agent on SwimmerGather, run:
```
python examples/trpo_emi_mujoco.py
```

To train an EMI agent on SparseHalfCheetah, run:

python examples/trpo_emi_mujoco.py --env=SparseHalfCheetah

To train an EMI agent on Montezuma's Revenge, run:
```
python examples/trpo_emi_atari.py
```
The first run will end with no operations other than creating a config. Run the command again if you see the configuration message.

Acknowledgements

This work was partially supported by Samsung Advanced Institute of Technology and Institute for Information & Communications Technology Planning & Evaluation (IITP) grant funded by the Korea government (MSIT) (No.2019-0-01367, BabyMind).

License

MIT License

snu-mllab/EMI