sinanuozdemir/maven-modern-nlp

Modern Natural Language Processing with GPT-3 & more

Introduction to Language Modeling with BERT, GPT3, and more!

This repository contains materials for our class on language modeling.

To use the code, clone the repository and install the required packages specified in the requirements.txt file. The code has been tested with Python 3.7 and above.

Lesson 1: Intro to LLMs and Getting Started with BERT

Slides

• Introduction to Language Modeling
• Introduction to BERT

Code

• Off-the-Shelf BERT
• Finetuning BERT for Classification
  • Data: Disaster detection From Kaggle
• BERT for Information Retrieval

Lesson 2: Querying Conversations with FLAN-T5

Mapping out a prototype with a mind for production

Slides

• Training + Fine-tuning in PyTorch

Code

• Conversational FLAN-T5

Lesson 3: Introduction to OpenAI

Slides

• GPT Attention + Parameters

Code

• OpenAI Prompting 101

Lesson 4: Advanced OpenAI Usage

Data

Sample OpenAI Fine-tuning data

Code

• OpenAI Prompting 201
• OpenAI Embeddings 101
• OpenAI Finetuning 101