/Ensemble-for-Generative-Models

REGEN: Recurrent Ensemble Methods for Generative Models

Primary LanguageJupyter Notebook

REGEN: Recurrent Ensemble Methods for Generative Models*

  • Paper Submission 준비중
  • 프로젝트 README.md 파일 수정 중

📖 Table of Contents

Table of Contents
  1. ➤ About The Paper
  2. ➤ Overview
  3. ➤ Project Files Description
  4. ➤ Getting Started
  5. ➤ References

📝 About The Paper

🌞 Overview

In this paper, we propose Recurrent Ensemble methods to take advantage of the structural characteristics of recurrent network models. Our proposed algorithms make an agreement at each time step and provide it to all constituent models for the following time step recurrently. To prove effectiveness our method, we experimented various recurrent network models in two tasks(Nerual mahcine Translation and String Arithmetic). Finally we verified that our methods outperform not only single model but also traditional ensembles.

💾 Project Files Description

  1. Neural Machine Translation
    2. Experiment 1: Translation spainish ➡ english
    • nmt_Training.ipynb - Train NMT single models
    • nmt_single.ipynb - Translate single model
    • nmt_esb_softvoting.ipynb - Translate using ensemble with Soft Voting
    • nmt_esb_survival.ipynb - Translate using ensemble with Survival
    • nmt_esb_majority.ipynb - Translate using ensemble with Majority
    • calcualte_TQE.ipynb - Evaluate Translation Quality for all ensemble models and single model
    • get_graph.ipynb - Graph for results
  2. String Arithmetic
    3. Experiments 2: Arithmetic as String
    • train_single.py - Train/Evaluate String Arithmetic single model
    • train_esb_softvoting.py - Train/Evaluate String Arithmetic ensemble with Soft Voting
    • train_esb_survival.py - Train/Evaluate String Arithmetic ensemble with Survival
    • train_esb_majority.py - Train/Evaluate String Arithmetic ensemble with Majority
    • get_graph.py - Graph for results

Some other supporting files

  1. Neural Machine Translation
    • dataset - Using Spanish - English dataset with random shuffle.
    • TQE(5-Epoch).xlsx - All Results for our experiments(source, target sentence and TQE per 2 epoch)
    • Checkpoint - Training takes too much time. You can download checkpoint in here.
  2. String Arithmetic
    • dataset - string arithmetic dataset consisted of 3 digit & 1 operator
    • common - base model, functions, layers, trainer etc.
    • training_memo - Training memo per epoch for all models(single, ensemble with soft voting, survival and majority)

📖 Getting Started

To clone and run this code, you'll need Git installed on your computer. From your command line:

# Clone this repository
$ git clone https://github.com/aaajeong/Recurrent-Ensemble.git

# Go into the repository
$ cd Recurrent-Ensemble

# Create Environment
$ conda env create -f recur_esb.yaml
$ conda activate recur_esb
  1. Neural Machine Translation
    2. You are able to train/test code by running jupyter notebook.
    You can also evaluate TQE by running jupyter notebook.
PyTorch 1.2.0 or higher is recommended. If the install below gives an error, please install pytorch first.
  2. String Arithmetic

    3. You can train/test the code by running like this: 

    $ python train_single.py

📖 References

Yasuki Saito, Deep Learning from scratch 2, hanbit(July 2018), chapter 07
Tensorflow Tutorials: Neural mahcine translation with Attention