Building off Fine-tuning 6-Billion GPT-J (& other models) in colab with LoRA and 8-bit compression This notebook is a simple example for fine-tuning GPT-J-6B with limited memory. A detailed explanation of how it works can be found in this model card. It is heavily based on this Colab. Huge thanks to Hivemind!
You can also finetune GPT-Neo-2.7B, French GPT-J (Cedille's Boris) and T0-3B with limited memory.
original colab: @gustavecortal
this repository contains a google colab and an archive of the finetuned model creatied from it