Demonstration of the Application of the BERT Algorithm for Feature Engineering using Keras:

Files:

  1. transformers_tokenizer.py:
  • Tokenize text
  • Prepare indices
  • Prepare segments
  1. transoformers_training/py:
  • Create sample sentence pairs
  • Build a token dictionary
  • Prepare the keras model
  • plot loss output

Sample Output:

alt text