OpenLMLab/MOSS-RLHF

Issues

• 第二篇论文中奖励模型训练的问题

  #58 opened 2 months ago by Syaoran1
  0

• 关于rm模型训练策略与损失函数

  #43 opened 10 months ago by tonylin52
  12

• Issue when merging llama with diff to generate English policy model

  #57 opened 3 months ago by foxlf823
  0

• 关于 root square of kl divs 与 rewards 的线性关系

  #56 opened 5 months ago by shirosheep000
  0

• 关于rm中lm loss计算的疑问

  #48 opened 9 months ago by DZ9
  1

• Has anyone compared this training framework to TRL?

  #54 opened 8 months ago by StarrySeas1
  1

• RM数据构造

  #55 opened 8 months ago by tcxia
  1

• 请问下代码里的kl散度问题

  #35 opened a year ago by rigorosyangffff
  1

• 关于ppo阶段，reward分数计算的问题

  #26 opened a year ago by mengyanggithub
  5

• 论文中提到在PPO流程中可以固定其他模型，先训练reward model直到value loss为0，请问这边具体是怎么进行训练的呢？

  #52 opened 8 months ago by HCHCXY
  1

• 对第二篇论文中有些不明白的地方请教解惑

  #53 opened 8 months ago by Obr00007576
  0

• 论文中rm对比学习训练方法疑问

  #45 opened 10 months ago by yhhh777
  4

• Part2中meta dataset的生成

  #51 opened 9 months ago by yata0
  1

• 训练集量级咨询

  #50 opened 9 months ago by Macvh
  1

• PPOSFTDataset bug report和相关问题咨询

  #49 opened 9 months ago by DZ9
  1

• 关于中文reward-model参数合并的问题

  #24 opened a year ago by hannlp
  4

• bash train_ppo_en.sh error

  #46 opened 10 months ago by robotzheng
  4

• Issues with using the released hh dataset.

  #44 opened 10 months ago by jltchiu
  2

• Clarification on MetaRM-optimization Implementation

  #42 opened 10 months ago by Benjamin-eecs
  2

• 请问目前支持基座模型使用Mistral-7b吗

  #39 opened 10 months ago by YijuGuo
  1

• [Question] Adaptive Margin

  #40 opened 10 months ago by eyuansu62
  3

• 自有的底座模型，自有的SFT权重，重新训练RM，可行么

  #38 opened a year ago by camposs1979
  1

• 关于reward model的部分的part 2有计划时间节点吗

  #31 opened a year ago by SpongebBob
  13

• Inference with SFT and Policy EN models

  #36 opened a year ago by henrypapadatos
  1

• Why are you not releasing your reward model for english?

  #37 opened a year ago by AmanSinghal927
  1

• 训练reward model的脚本

  #16 opened a year ago by wangzhao88
  3

• 合并权重问题

  #34 opened a year ago by red-tie
  6

• 关于reward model的权重合并问题

  #33 opened a year ago by HuipengXu
  1

• 资源占用问题

  #32 opened a year ago by Ming-Di
  3

• 关于配置环境

  #23 opened a year ago by zjutkarma
  2

• Technical report PART 2

  #13 opened a year ago by snowkcon
  3

• 关于reward model

  #10 opened a year ago by skepsun
  5

• Any benchmark vs SFT？

  #30 opened a year ago by guotong1988
  2

• Training on 8 Nvidia RTX A6000

  #19 opened a year ago by Top34051
  1

• PPO data en

  #27 opened a year ago by borisshapa
  1

• deepspeed的parameter_offload问题

  #29 opened a year ago by LiangZhuuu
  1

• PPO显存占用问题

  #28 opened a year ago by LiangZhuuu
  0

• typo

  #25 opened a year ago by chosenone75
  1

• 关于Reward model打分的一些疑惑

  #21 opened a year ago by hannlp
  12

• reward model训练的哪些方面的能力

  #22 opened a year ago by yuanhuachao
  1

• 英文的PPOdata

  #20 opened a year ago by QYHcrossover
  1

• 内存占用大问题

  #12 opened a year ago by QYHcrossover
  2

• value model与reward model

  #18 opened a year ago by KUANWB
  2

• PPO训练稳定性问题

  #17 opened a year ago by hust-kevin
  5

• Reward Model

  #11 opened a year ago by Cyber-Axe
  2

• Training script of reward model

  #14 opened a year ago by zwhe99
  2

• reward_model准确率

  #15 opened a year ago by mingrenbuke
  1

• support lora training

  #9 opened a year ago by akk-123
  1

• 用于PPO训练的数据结构

  #7 opened a year ago by Arain-sh
  2

• Can I run this pipeline on A100-40GB?

  #8 opened a year ago by zwhe99
  4