/DiSAN-pytorch

DiSAN: Directional Self-Attention Network for RNN/CNN-Free Language Understanding

Primary LanguagePython

DiSAN: Directional Self-Attention Network for RNN/CNN-Free Language Understanding

Pytorch re-implementation of DiSAN: Directional Self-Attention Network for RNN/CNN-Free Language Understanding.

This is an unofficial implementation. There is the implementation by the authors, which is implemented on Tensorflow.

Results

Dataset: SNLI

Model Valid Acc(%) Test Acc(%)
Baseline from the paper - 85.62
Re-implemenation 80.60 79.80

Development Environment

  • OS: Ubuntu 16.04 LTS (64bit)
  • Language: Python 3.6.6
  • Pytorch: 0.4.0

Requirements

Please install the following library requirements first.

nltk==3.3
tensorboardX==1.2
torch==0.4.0
torchtext==0.2.3

Training

python train.py --help

    usage: train.py [-h] [--batch-size BATCH_SIZE] [--data-type DATA_TYPE]
                    [--dropout DROPOUT] [--epoch EPOCH] [--gpu GPU]
                    [--learning-rate LEARNING_RATE] [--print-freq PRINT_FREQ]
                    [--weight-decay WEIGHT_DECAY] [--word-dim WORD_DIM]
                    [--d-e D_E] [--d-h D_H]

    optional arguments:
      -h, --help            show this help message and exit
      --batch-size BATCH_SIZE
      --data-type DATA_TYPE
      --dropout DROPOUT
      --epoch EPOCH
      --gpu GPU
      --learning-rate LEARNING_RATE
      --print-freq PRINT_FREQ
      --weight-decay WEIGHT_DECAY
      --word-dim WORD_DIM
      --d-e D_E
      --d-h D_H

Note:

  • Only codes to use SNLI as training data are implemented.
  • There must be some erros somewhere, as the performance is much lower. But I couldn't find...