What is this tutorial?
I am writing this tutorial specifically for a Natural Language Processing class at Georgia Tech, to ease into a problem set I wrote for the class on deep transition parsing. The problem set uses some advanced techniques. The intention of this tutorial is to cover the basics, so that students can focus on the more challenging aspects of the problem set.
There are plenty of other tutorials out there, but they all seem to have one of two problems, which is why I am writing my own:
- They have a lot of content on computer vision and conv nets, which is irrelevant for our class and might just confuse students
- Pytorch is brand new, and so many deep learning for NLP tutorials are in older frameworks, and usually not in dynamic frameworks like Pytorch, which have a totally different flavor.
Table of Contents:
- Introduction to Torch's Tensor Library
- Computation Graphs and Automatic Differentiation
- Deep Learning Building Blocks: Affine maps, non-linearities, and objectives
- Optimization and Training
- Creating Network Components in Pytorch
- Example: Logistic Regression Bag-of-Words text classifier
- Word Embeddings: Encoding Lexical Semantics
- Example: N-Gram Language Modeling
- Exercise: Continuous Bag-of-Words for learning word embeddings
- Sequence modeling and Long-Short Term Memory Networks
- Example: An LSTM for Part-of-Speech Tagging
- Exercise: Augmenting the LSTM tagger with character-level features
- Advanced: Making Dynamic Decisions
- Example: Bi-LSTM Conditional Random Field for named-entity recognition
- Exercise: A new loss function