hasibzunair/generative-models-tutorial

Tutorial materials for Ericsson AI & ML Upskill Program 2023-2024.

Building and Applying Generative Models using PyTorch

Tutorial materials for Ericsson AI & ML Upskill Program 2023-2024. It contains code for finetuning a GPT-like large language model (LLM) in PyTorch.

Materials for the previous program can be found Ericsson AI & ML Upskill Program 2021.

Learning resources

• Intro to Large Language Models
• Understanding Large Language Models
• QLoRA is all you need (Fast and lightweight model fine-tuning)
• Parameter-efficient Finetuning with LoRA
• QLoRA: Efficient Finetuning of Quantized LLMs
• Run Mixtral-8x7B models in Colab or consumer desktops

Acknowledgements

This repository was built using Implementing a ChatGPT-like LLM in PyTorch from scratch, step by step.