/transformer-xl1

Primary LanguagePython

Project Name <--- FIXME

Library Logo <--- FIXME

Description about your project. Why do you choose to build this? <--- FIXME

Slide about your project (if it's available) <--- FIXME

Architecture Image <--- FIXME

Authors:

  • Github: members github name <--- FIXME
  • Email: members emails <--- FIXME

Advisors:

  • Github: advisor github name <--- FIXME
  • Email: advisor emails <--- FIXME

I. Set up environment

  • Step 1: <--- FIXME
conda env create -f environment.yml
  • Step 2: <--- FIXME
  • Step 3: <--- FIXME

II. Set up your dataset

  • Guide user how to download your data and set the data pipeline <--- FIXME
  • References: NLP and CV

III. Training Process

FIXME

Training script:

python train.py --epochs ${epochs} --input-lang en --target-lang vi --input-path ${path_to_en_text_file} --target-path ${path_to_vi_text_file}

FIXME

Example:

!python train.py --train-folder ${train_folder} --valid-folder ${valid_folder} --num-classes 2 --patch-size 5 --image-size 150 --lr 0.0001 --epochs 200 --num-heads 12

FIXME

There are some important arguments for the script you should consider when running it:

  • train-folder: The folder of training data
  • valid-folder: The folder of validation data
  • ...

IV. Predict Process

python predict.py --test-data ${link_to_test_data}

V. Result and Comparision

FIXME

Your implementation

Epoch 7/10
782/782 [==============================] - 261s 334ms/step - loss: 0.8315 - acc: 0.8565 - val_loss: 0.8357 - val_acc: 0.7978
Epoch 8/10
782/782 [==============================] - 261s 334ms/step - loss: 0.3182 - acc: 0.8930 - val_loss: 0.6161 - val_acc: 0.8047
Epoch 9/10
782/782 [==============================] - 261s 333ms/step - loss: 1.1965 - acc: 0.8946 - val_loss: 3.9842 - val_acc: 0.7855
Epoch 10/10
782/782 [==============================] - 261s 333ms/step - loss: 0.4717 - acc: 0.8878 - val_loss: 0.4894 - val_acc: 0.8262

FIXME

Other architecture

Epoch 6/10
391/391 [==============================] - 115s 292ms/step - loss: 0.1999 - acc: 0.9277 - val_loss: 0.4719 - val_acc: 0.8130
Epoch 7/10
391/391 [==============================] - 114s 291ms/step - loss: 0.1526 - acc: 0.9494 - val_loss: 0.5224 - val_acc: 0.8318
Epoch 8/10
391/391 [==============================] - 115s 293ms/step - loss: 0.1441 - acc: 0.9513 - val_loss: 0.5811 - val_acc: 0.7875

Your comments about these results <--- FIXME

VI. Running Test

When you want to modify the model, you need to run the test to make sure your change does not affect the whole system.

In the ./folder-name (FIXME) folder please run:

pytest