Practical, short, batched, gpu-ready, self-contained implementations of various NLP tasks using Pytorch 1.3.0
Names of different nationalities are taken from Pytorch tutorial: https://pytorch.org/tutorials/intermediate/seq2seq_translation_tutorial.html Thai names: https://hearnames.com/pronunciations/thai-names/thai-surnames Translation pairs: https://www.manythings.org/anki/
name_classifier_rnn_onehot.py
- Classify texts using RNN with one hot encoded vectors as inputs.
name_classifier_rnn_embedding.py
- Classify names using RNN with dense embeddings.
name_classifier_cnn_onehot.py
Convolutional Neural Networks for Sentence Classification
- Classify texts using CONV1D layers.
- Using one hot encoded vectors.
name_generator.py
- Taking name prefixes and matching it with the next letter. Easy to understand implementation.
name_generator_optimized.py
- Targets are mapped to input packed sequence order. Uses less samples, therefore is faster and memory efficient.
seq2seq_onehot.py
Sequence to Sequence Learning with Neural Networks
- Latest hidden state of the encoder RNN is passed as initial state of the decoder RNN.
- This implementation is using teacher forcing and one hot encoded vectors.
seq2seq_unforced_onehot.py
Scheduled Sampling for Sequence Prediction with Recurrent Neural Networks
- Latest hidden state of the encoder RNN is passed as initial state of the decoder RNN.
- Tunable teacher forcing for the decoder allows to use model outputs during training.
- Using one hot encoded vectors.
seq2seq_propagated_onehot.py
Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation
- Latest hidden state of the encoder RNN is passed as initial state of the decoder RNN as well as additional context to be combined with every hidden state and output.
- Using teacher forcing and one hot encoded vectors.
seq2seq_attn_onehot.py
Neural Machine Translation by Jointly Learning to Align and TranslateEffective Approaches to Attention-based Neural Machine Translation
- All hidden states of the encoder RNN is passed to the decoder RNN.
- Decoder RNN choses what hidden state to use when generating output, therefore removing the bottleneck introduced by encoder output vector size.
- Using one hot encoded vectors.