/diy.ai

Primary LanguagePython

DIY AI

This repo contains my tutorial and implementation for ML,DL,LLM,NLP and ASR, ranging from ML basics to advanced genAI, from algorithm to system, and from MLOps to AI on the edge. This repo is primarily based on github, Pytorch, Huggingface🤗, and Colab. And I appreciate great material online like Stanford's Machine Learning, Mu Li's Dive into deep learning and Hungyi Lee's genAI.

Math Prerequites

I found these great cheatsheets for beginner & refresher:

ML Basics

Follow the coding practice by google.

See implementation details.

GenAI - Generative AI

Salute to Prof. Lav Varshney for his awesome genAI course.

  • VAE - Variational autoencoder

  • GAN - Generative adversarial network

  • Autoregressive

  • Diffusion

    Huggingface🤗 diffusion class

    Huggingface🤗 diffusers

  • Transformer

    Huggingface🤗 transformers

  • PEFT - Parameter-Efficient Fine-Tuning

    1. Huggingface🤗 peft
    2. Lora:Check out their paper and source

  • Prompt

  • Detection

NLP - Natural Language Processing

  • GPT - Generative Pre-trained Transformer The best materials are definitely Andrej Karpathy with his build GPT from sratch. And he does have several versions of minimal GPTs:

    1. minGPT for education: A minimal PyTorch re-implementation of the OpenAI GPT (Generative Pretrained Transformer) training

    2. nanoGPT with teeth: The simplest, fastest repository for training/finetuning medium-sized GPTs.

    3. LLM101n his latest tutorial thats still under construction.

    I would personally start with minGPT and his youtube to get a sense of it, get hands dirty and use the latter two as reference.

  • BERT - Bidirectional Encoder Representations from Transformers

ASR - Automatic Speech Processing

  • Audio Preprocessing

    Look for tag Audio in the tutorials.

    Also the torchaudio's official tutorial.

  • CTC - Connectionist Temporal Classification

  • RNNT - RNN Transducer

  • SSL - Wav2Vec2.0

  • WSL - Whisper

EdgeAI - AI on the edge

  • Transformer.js

    See the doc and Source

  • Llama.cpp

    Inference of Meta's LLaMA model (and others) in pure C/C++. See their source.

  • llm.c

    LLM training in simple, raw C/CUDA. See their source.

ML Sys

Shout out to Prof.Tianqi Han's ML Sys on how to build and optimize ML systems, and Stanford's CS129 and UIUC's ECE408 on paralell computing.

  • Intro to CUDA

    See my notes and code on ECE408, a great course for learning CUDA.

  • vLLM

  • FlashAttention

  • DeepSpeed

  • ML Ops

Application

  • Codegen

    source

  • Whisper WebGPU

    app: ML-powered speech recognition directly in your browser

  • Voice Assistant

    source: Siri + llm inference + Cloudflare worker

  • Naming

    GPT+RAG for classficial Chinese name

  • AIMusic