Keras-MMoE

This repo contains the implementation of Multi-gate Mixture-of-Experts model in Keras.

Here's the video explanation of the paper by the authors.

The repository includes:

A Python 3.6 implementation of the model in Keras with TensorFlow backend
- The code is also compatible with Python 2.7
Example demo of running the model with the census-income dataset from UCI
- This dataset is the same one in Section 6.3 of the paper

The code is documented and designed to be extended relatively easy. If you plan on using this in your work, please consider citing this repository (BibTeX is included below) and also the paper.

Getting Started

Requirements

Python 3.6
Other libraries such as TensorFlow and Keras listed in requirements.txt

Installation

Clone the repository
Install dependencies

pip install -r requirements.txt

Run the example code

python census_income_demo.py

Notes

Due to ambiguity in the paper and time and resource constraints, we unfortunately can't reproduce the exact results in the paper

Contributing

Contributions to this repository are welcome. Examples of things you can contribute:

Performance improvements by re-writing the model in TensorFlow, PyTorch, or MXNet
Improve the census income benchmark to be as close as the one from the paper
Improve the synthetic benchmark to be as close as the one from the paper
Training on other public datasets for different benchmarks
Accuracy improvements
Visualizations

Citation

Use this BibTeX to cite the repository:

@misc{keras_mmoe_2018,
  title={Multi-gate Mixture-of-Experts model in Keras and TensorFlow},
  author={Deng, Alvin},
  year={2018},
  publisher={Github},
  journal={GitHub repository},
  howpublished={\url{https://github.com/drawbridge/keras-mmoe}},
}

Acknowledgments

The code is built upon the work by Emin Orhan.

hwenjun18/keras-mmoe