This tutorial is an introduction of using Deep Learning algorithm in the domain of Natural Language Processing.
And it is prepared using content (theory and code) from following sources:
- Deep Learning with Python, Book by François Chollet
- Neural Network Methods in Natural Language Processing, Book by Yoav Goldberg
- CS224d: Deep Learning for Natural Language Processing
Practice code on Kaggle's Toxic Comment Classification Challenge dataset
-
- Sequence classification
- Language detection
- Category classification (Sentiment, topics etc.)
- Keyword classification (name-gender, place/person name)
- Sequence to sequence (Seq2Seq)
- Translation
- Gmail smart reply
- Conversational AI: Chat bots
- Others
- Name, Story, poem, dialog generator
- Image captioning
- Part of speech tagging
- Name entity recognition
- Sequence classification
-
- Python 3.6
- pip
- Virtualenv
- Libraries:
- Keras
- Tensorflow
- Jupyter
- matplotlib
-
- IMDB review dataset
- Kaggle (Toxic comment classification challenge) Wikipedia comment dataset
- Ubuntu dialog corpora
- Translation dataset
- Other datasets
-
- General Analysis
-
- Representation
- One Hot Encoding
- Word Embeddings
- Pre trained embeddings
- Word2vec
- GloVe
- Pre trained embeddings
-
Models
- Embedding to Class Model 1
- Embedding connected to 1 layer RNN (Recurrent Neural Network) Model 2 and Model 2 Extended
- Bidirectional RNN Model 3 and Model 3 Extended
-
- Long short-term memory (LSTM)
- Gater Recurrent Unit (GRU)
- Seq2seq
- Attention
- Beam Search
-
Keras
- API & keywords
- Optimizers
- Loss
- Activation
- Metrics
- Deploy model to production and inference
- API & keywords
-
Model optimization techniques
- Dropout
- Truncated backpropagation through time (TBPTT)
- Vanishing Gradient Problem