kongds/MoRA

Issues

• 数据集格式

  #20 opened 2 months ago by lcykww
  1

• RuntimeError: The size of tensor a (4096) must match the size of tensor b (256) at non-singleton dimension 1

  #19 opened 2 months ago by lcykww
  2

• The experiments on Instruction Tuning and Continual Pretraining

  #13 opened 6 months ago by lucasliunju
  43

• 可以提供复现UUID的数据集和实验配置吗

  #18 opened 4 months ago by 2018211801
  2

• 运行时报错：MoRA/peft-mora/src/peft/tuners/lora /layer.py文件中有俩个小bug

  #17 opened 4 months ago by chen-c-alt
  1

• How to run the fine-tuning using slurm?

  #16 opened 4 months ago by AlessioQuercia
  1

• ReMoRa + DoRa improves on ReMoRa

  #10 opened 7 months ago by catid
  2

• Please add a license to this repo

  #15 opened 5 months ago by mbrukman
  2

• MoRA, type:6 / RuntimeError: mat1 and mat2 shapes cannot be multiplied

  #14 opened 5 months ago by daebum1994
  1

• support QMoRA?

  #12 opened 6 months ago by ByungKwanLee
  1

• Broken model output

  #11 opened 6 months ago by kallewoof
  9

• Incorrect LoRa init

  #9 opened 7 months ago by catid
  3

• Integrate into torchtune

  #8 opened 7 months ago by impredicative
  4

• Document mora_type

  #6 opened 7 months ago by winglian
  2

• unused code

  #7 opened 7 months ago by winglian
  1

• Steps to run the code and optimal parameters for continual pretraining

  #5 opened 7 months ago by gs7vik
  2

• About the baselines

  #4 opened 7 months ago by lucasliunju
  5

• Add Arxiv paper link to readme

  #3 opened 7 months ago by impredicative
  1

• Push to Hub

  #1 opened 7 months ago by alielfilali01
  4