BorealisAI/flora-opt

This is the official repository for the paper "Flora: Low-Rank Adapters Are Secretly Gradient Compressors" in ICML 2024.

PythonLGPL-3.0

Issues

Support 7b model fine-tuning with DDP
#5 opened 5 months ago by seongjunyun
5
Flora without FloraAccelerator
#4 opened 5 months ago by rockerBOO
5
Clean install of `flora-opt[torch]` fails import
#3 opened 5 months ago by atong01
4
Implementing Flora with Adam
#1 opened 5 months ago by aashiqmuhamed
2
Question regarding LOMO
#2 opened 6 months ago by Neukiru
3