Xin-Zhou-smu/Seq2SeqDiverse

Seq2Seq model implemented with pytorch, using temperature sampling, top-k sampling, top-p sampling / nucleus sampling, and beam search.

Seq2Seq with Diverse Decoding Algorithm

This is a Seq2Seq model with multiple diverse decoding algorithms.

Datasets:

• dataset1: Quora

Models:

• model1: Seq2Seq

Decoding Algorithms:

• Temperature Sampling

• Top-K Sampling

• Top-P Sampling / Nucleus Sampling

• Beam Search

• Diverse Beam Search (TODO)

Data Process

PYTHONPATH=. python dataprocess/process.py

Unit Test

• for loader

PYTHONPATH=. python loaders/loader1.py

• for module

PYTHONPATH=. python modules/module1.py

Main Process

python main.py

You can change the config either in the command line or in the file utils/parser.py

Here are the examples:

# train
python main.py \
    --mode train

# test with temperature_sampling
python main.py \
    --mode test \
    --decoding_algorithm temperature_sampling \
    --T 1e-13

# test with top_k_sampling
python main.py \
    --mode test \
    --decoding_algorithm top_k_sampling \
    --K 10

# test with top_p_sampling
python main.py \
    --mode test \
    --decoding_algorithm top_p_sampling \
    --P 0.3

# test with beam_search
python main.py \
    --mode test \
    --decoding_algorithm beam_search \
    --B 4