timothyxp/Lora

Integration Lora with HuggingFace Library and test it on tweets dataset

Lora

Test Lora with HuggingFace Library and test it on tweets dataset In notebook you can find function for automatic replace Linear layers to Lora layers and example of fine-tuning it for tweets dataseet, because no such examples existing in original repo

Original Lora repo - https://github.com/microsoft/LoRA

Logs of experiments with RoBerta and Deberta - https://wandb.ai/timothyxp/Lora?workspace=user-timothyxp