howsmyanimeprofilepicture/DiffusionFromScratch

Rebuild the Stable Diffusion Model in a single python script. Tutorial for Harvard ML from Scratch Series

Diffusion From Scratch

Binxu Wang (binxu_wang@hms.harvard.edu)

Tutorial on Stable Diffusion Models at ML from Scratch seminar series at Harvard.

• Homepage
• Tutorial Slides

This tiny self-contained code base allows you to

• Rebuild the Stable Diffusion Model in a single Python script.
• Train your toy version of stable diffusion on classic datasets like MNIST, CelebA

Colab notebooks

• Playing with Stable Diffusion and inspecting the internal architecture of the models. Open in Colab
• Build your own Stable Diffusion UNet model from scratch in a notebook. (with < 300 lines of codes!) Open in Colab
  • Self contained script
  • Unit tests
• Build a Diffusion model (with UNet + cross attention) and train it to generate MNIST images based on the "text prompt". Open in Colab (exercise) Open in Colab (answer)

Demo Outputs

Music video generated from Stable Diffusion.