machine-learning-for-nlp-guide

Guide for engineers interested in NLP machine learning

Path

Understand possibilities and form business applications
1. Everyone AI for Everyone
Either level up through:
1. Gaining theoretical foundation of Deep Learning for NLP
  1. Stanford Course Materials http://web.stanford.edu/class/cs224n/
  2. Natural Language Processing with Deep Learning https://www.youtube.com/watch?v=8rXD5-xhemo&list=PLoROMvodv4rOhcuXMZkNm7j3fVwBBY42z
  3. Stanford CS224U: Natural Language Understanding https://www.youtube.com/watch?v=tZ_Jrc_nRJY&list=PLoROMvodv4rObpMCir6rNNUlFAn56Js20
2. Getting "Practical" Knowledge of Deep Learning for NLP
Learn how to Deep Learning
1. Nuts and Bolts of Applying Deep Learning
2. "Everyday" Engineers Fast.ai
3. Research Engineers Deep Learning AI
Learn about all the stuff "they don't teach"
1. Learn Production-Level Deep Learning: https://fullstackdeeplearning.com/
2. Resources: https://github.com/full-stack-deep-learning/fsdl-text-recognizer-project
Base Models to Use
1. Spacy for general NLP tasks
2. HuggingFace Transformers
Profit

State of the Art Methods

Resources

Syntactic Search over Wikipedia: https://spike.wikipedia.apps.allenai.org/search/wikipedia
Odinson: Rapidly query a natural language knowledge base https://github.com/lum-ai/odinson
CheckList: Behavioral Testing NLP https://github.com/marcotcr/checklist
Data project checklist https://www.fast.ai/2020/01/07/data-questionnaire
BERT, ELMo, & GPT-2: How Contextual are Contextualized Word Representations? http://ai.stanford.edu/blog/contextual/
BERT commit log https://amitness.com/2020/05/git-log-of-bert/
Full stack deep learning github repo: https://github.com/full-stack-deep-learning/fsdl-text-recognizer-project
Expand Data Labeled Data using Unlabled Data
- Blog: https://ai.googleblog.com/2019/03/harnessing-organizational-knowledge-for.html
- Detailed Article: https://towardsdatascience.com/a-look-into-snorkel-drybell-8e9e781dc250
Explain Predictions
- Python Library: https://github.com/jphall663/awesome-machine-learning-interpretability
Deploy models to production
- Tutorial: https://hackernoon.com/enterprise-af-solution-for-text-classification-using-bert-9fe2b7234c46
Learn how to implement new models
- Deep Learning from the Foundations: https://www.fast.ai/2019/06/28/course-p2v3/
More Learning Resources:
- The Best Artificial Intelligence, Machine Learning and Data Science Resources*
nlp-library curated list of papers
Machine Learning System Best Practice and Design:
- The ML Test Score: A Rubric for ML Production Readiness and Technical Debt Reduction: https://ai.google/research/pubs/pub46555
- Machine Learning: The High Interest Credit Card of Technical Debt: https://ai.google/research/pubs/pub43146
An Interactive Visualization to Explore NLP Papers
How Big Should My Language Model Be?
Accelerate your NLP pipelines using Hugging Face Transformers and ONNX Runtime

Tools

https://prodi.gy/buy
- Text and image annotation
https://github.com/chakki-works/doccano
- Open source text annotation tool
https://www.media.mit.edu/projects/dive/overview/
- DIVE is a web-based data exploration system that lets non-technical users create stories from their data without writing code. DIVE combines semantic data ingestion, recommendation-based visualization and analysis, and dynamic story sharing into a unified workflow.