计算r_loss时，似乎会把padding_idx也考虑进去，如果output和bad_output长度相差太大会对结果有较大影响吗？

Opened this issue 5 months ago · 0 comments

world1tree commented 5 months ago

TIM/sft_reward_training/trainer/utils/model/reward_model.py

Line 167 in 5876473

end_ind = max(c_ind, r_ind)