/tf2-transformer-chatbot

Transformer Chatbot in TensorFlow 2.1

Primary LanguageJupyter NotebookMIT LicenseMIT

Transformer Chatbot with TensorFlow 2.1

Build an end-to-end chatbot with Transformer in TensorFlow 2.1. Checkout the my tutorial on blog.tensorflow.org.

Updates

  • 18 Jan 2019: Added notebook with Google Colab TPU support in TensorFlow 2.1.

Packages

pip install -r requirements.txt

Dataset

  • We will use the conversations in movies and TV shows provided by Cornell Movie-Dialogs Corpus, which contains more than 220 thousands conversational exchanges between more than 10k pairs of movie characters, as our dataset.
  • We pre-process our dataset in the following order:
    • Extract max_samples conversation pairs into list of questions and answers.
    • Pre-process each sentence by removing special characters in each sentence.
    • Build tokenizer (map text to ID and ID to text) using TensorFlow Datasets SubwordTextEncoder.
    • Tokenize each sentence and add start_token and end_token to indicate the start and end of each sentence.
    • Filter out sentence that has more than max_length tokens.
    • Pad tokenized sentences to max_length
  • Check dataset.py implementation.

Model

transformer model plot

  • check model.py for the implementation of Multi-Headed Attention, Positional Encoding and Transformer.

Run

  • check all availble flags and hyper-parameters python main.py --help
python main.py --batch_size 256 --epochs 50 --max_samples 50000

Samples

input:		where have you been?
output:		i m not talking about that .

input:		it's a trap!
output:		no , it s not .