/LoRA_implementation

This is the implementation of low rank adaptation (LoRA) which is a subset of parameter efficient fine tuning (PEFT).

Primary LanguageJupyter Notebook

Low Rank Adaptation (LoRA) implementation

This is the implementation of low rank adaptation (LoRA) which is a subset of parameter efficient fine tuning (PEFT).

Papers references:

LoRA

Effectiveness of Language Model Fine-Tuning

Intrinsic Dimensionality Paper