TencentARC/LLaMA-Pro

Issues

• finetune_cosmopedia.sh如何训练出来8B模型

  #27 opened 6 months ago by RuipingWang1986
  1

• 关于论文中通用能力榜单几乎没有下降，部分反而有提升

  #31 opened 5 months ago by bestpredicts
  1

• 请教下训练的显存需求

  #20 opened 8 months ago by denghj3
  6

• 这个方法可以扩展到vit类的视觉encode上吗？

  #33 opened 3 months ago by lucasjinreal
  0

• 请教大佬可以训练qewn2-7b吗

  #32 opened 3 months ago by jqtian123
  0

• 关于运行流程

  #30 opened 5 months ago by GOOD-N-LCM
  4

• 关于零初始化和扩展层的位置

  #28 opened 5 months ago by ouyanxi1125
  4

• 训练到10B tokens 时loss就收敛了 无法下降

  #29 opened 5 months ago by bestpredicts
  1

• 利用finetune_cosmopedia.sh脚本进行继续预训练中的数据集如何构建

  #26 opened 6 months ago by RuipingWang1986
  2

• Thanks for wonderful projects ! Why I always got the results of apparent loss of original ability?

  #25 opened 6 months ago by hzgdeerHo
  8

• 增量预训练的疑惑？

  #13 opened 9 months ago by zhuxiaobin
  6

• guide to run the code

  #11 opened 9 months ago by Abolfazl-kr
  2

• 请教下论文中的实验

  #24 opened 7 months ago by ChrisXULC
  1

• 论文Table7请教

  #1 opened 10 months ago by XiaoYee
  5

• Training on arbitary data

  #23 opened 7 months ago by HelloWorldLTY
  2

• Pretrain code of Mistral-Pro-8B-v0.1

  #22 opened 8 months ago by shawnricecake
  1

• Do we need to freeze embedding layer and the lm_head as well during the Llama-pro style training ?

  #21 opened 8 months ago by shamanez
  2

• 新增的transformer层是与上一层共享参数吗？

  #16 opened 8 months ago by CharlinChen
  3

• Comparison with PEFT

  #19 opened 8 months ago by LaVieEnRose365
  1

• 更大的模型需要更多的block吗？

  #18 opened 8 months ago by PoseidomWong
  1

• 您好，请教一下post pretrain的问题

  #10 opened 9 months ago by ray075hl
  8

• llama factory的llama-pro是不是写得不对啊

  #15 opened 9 months ago by HuXinjing
  2

• Question regarding the difference between llama-pro and the regular llama.（关于llama-pro和普通llama之间的区别的疑问）

  #9 opened 10 months ago by WUHU-G
  8

• 对比lora优势是什么

  #14 opened 9 months ago by xiaozhu1106
  1

• Issue with Model Saving After Layer Expansion: Removed Shared Tensors

  #12 opened 9 months ago by yumingfan-0219
  2

• 我们如何针对扩展区块微调?

  #3 opened 10 months ago by win10ogod
  5

• How to load the new model weight

  #8 opened 10 months ago by khalil-Hennara
  1

• Should I freeze norm.weight?

  #7 opened 10 months ago by metterian
  1

• Question about Llama-7B and Llama-7B-Pro comparison.

  #5 opened 10 months ago by ryusaeba
  2

• full code to continue pre-training

  #6 opened 10 months ago by Abolfazl-kr
  1

• Code for training llama pro?

  #2 opened 10 months ago by yhyu13
  8

• Arxiv Data

  #4 opened 10 months ago by ZhengTang1120
  2