This repository contains various experiments to understand and improve interpertability in gated modular neural networks. Currently I am using the various Mixture of Experts architecture models listed below for these experiments:
- Expectation Model
- Stochastic Model
- Pre-softmax Model
- EM Model
Python 3.7
Pytorch 1.6.0, optionally with Cuda 10.1
- Linux Operating System. It has been tested on Ubuntu and MacOS.
- Additional modules listed in
requirements.txt
In order to install the code locally please follow the steps below:
-
Clone this repository and go to the cloned directory.
-
Set the environment variable to point to your python executable:
export PYTHON=<path to python 3.7 executable>
-
Run the following command to set up the environment:
make
on Linux/Mac -
Activate the environment by running:
source mnn/bin/activate
on Linux/Mac
-
Run the following script to start jupyter:
./bin/run_notebooks.sh
-
In the jupyter lab go to the notebooks folder which contains all the relevant notebooks
-
Start with the toy_classification.ipynb.
-
Select the mnn kernel.
-
You should now be able to run the notebooks.
For any questions or issues email: yamuna dot krishnamurthy at rhul.ac.uk