leehanchung/lora-instruct

Falcon -7B training loss not reducing

pcvishak opened this issue · 0 comments

pcvishak commented

Thanks for the wonderful code. While training Falcon-7B on Alpaca dataset, the training loss is not reducing. It used to work fine. Were there any recent changes?