huggingface/trl

Issues

• RuntimeError: chunk expects at least a 1-dimensional tensor

  #2338 opened 15 days ago by imrankh46
  3

• [Question] `add_generation_prompt=True` on prompt

  #2346 opened 11 days ago by Galaxy-Husky
  5

• RLOO Checkpoint Issue

  #2342 opened 12 days ago by asparius
  1

• Several problems in RLOOTrainer

  #2316 opened 12 days ago by serendipity800
  0

• Whether chatglm3 6b is supported by trl ?

  #2299 opened 12 days ago by fjy01
  1

• RLooTrainer bug when using deepspeed

  #2329 opened 17 days ago by macheng6
  4

• Support for MiniCPM-V Reinforcement Learning with Direct Preference Optimization (DPO)

  #2326 opened 18 days ago by DarioPTWR
  2

• [Question] Why is Importance Sampling and Clipping applied in RLOO?

  #2341 opened 12 days ago by shashankg7
  0

• Accelerator package version problem

  #2335 opened 16 days ago by littleshutong
  2

• Multiple Errors with PPOTrainer. error in ppo_trainer.dataloader

  #2340 opened 13 days ago by Debolena7
  2

• DPO Training DataLoader is not shuffled

  #2337 opened 16 days ago by kaiwenw
  1

• Difference between SFTTrainer and Seq2seqTrainer

  #2339 opened 13 days ago by Hyfred
  0

• PPOConfig.__init__() got an unexpected keyword argument 'model_name'

  #2314 opened 15 days ago by chenyang399
  7

• trl dpo AttributeError: 'generator' object has no attribute 'generate'

  #2292 opened 25 days ago by MonolithFoundation
  9

• OOM when unwrap_model_for_generation

  #2250 opened a month ago by hlnchen
  3

• KTOTrainer Memory Leakage

  #2268 opened a month ago by Isaaclgz
  3

• Set gradient_checkpointing_kwargs in the yaml

  #2334 opened 16 days ago by Galaxy-Husky
  0

• Wrong tensor index for roll and truncate in DPOTrainer fn concatenated_forward( ).

  #2330 opened 16 days ago by yanghh2000
  1

• AttributeError: 'TrainingArguments' object has no attribute 'model_init_kwargs'

  #2291 opened 18 days ago by MonolithFoundation
  3

• OOM when finetuning Llama3.2-90B on 8xA100 80GB

  #2294 opened 24 days ago by maximilianmordig
  0

• Tries to iterate over Modules of value_model, even when it is NoneType (the default value)

  #2312 opened 17 days ago by ColinG03
  1

• RLOOTrainer ignores custom DataCollatorWithPadding in favor of default one

  #2309 opened 21 days ago by anch0vy
  1

• Using a different `ref_model` from `model` leads to incorrect results

  #2307 opened 21 days ago by DarshanDeshpande
  1

• Gather metrics on All GPUs

  #2315 opened 18 days ago by ziyuwan
  0

• save `training_args` in another file format

  #2313 opened 18 days ago by not-lain
  2

• Feature Request: String-Based Comparison Reward model for RLOOTrainer

  #2280 opened a month ago by HiroshigeAoki
  1

• IndexError: pop from an empty deque

  #2304 opened 18 days ago by c3ianwu
  2

• Code migration suggestions

  #2296 opened 24 days ago by MonolithFoundation
  1

• Documentation for PairRM Judge is not displaying correctly

  #2317 opened 18 days ago by LuisVasquezBSC
  1

• RLOOTrainer ignores custom DataCollatorWithPadding in favor of default one

  #2308 opened 21 days ago by staas-dnm
  1

• Conflict between last version of Transformers.Trainer and DPOTrainer.get_batch_samples

  #2275 opened a month ago by lucasdegeorge
  6

• DataCollatorForCompletionOnlyLM not working

  #2293 opened 25 days ago by SwayamInSync
  1

• The equvalence of "KL as loss" and directly optimization of KL?

  #2284 opened 25 days ago by serendipity800
  0

• "tokenizer" attribute is now deprecated on Transformers Trainer class

  #2290 opened 25 days ago by danib08
  1

• example/scripts/sft.py doesn't work

  #2253 opened a month ago by Reductionreaction
  3

• how much data needed when training 11B model, Llama-3.2-11B-Vision-Instruct?

  #2269 opened a month ago by cutecharmingkid
  1

• Add model merging callback

  #2241 opened a month ago by lewtun
  6

• Significant Difference between torchrun launch and accelerate launch

  #2262 opened a month ago by SinclairCoder
  0

• wrong objective/entropy in RLOOTrainer

  #2281 opened a month ago by serendipity800
  0

• During the execution of XPO, a 'tokenizer' KeyError suddenly occurred in callbacks.py

  #2264 opened a month ago by ArcherShirou
  3

• PPOv2Trainer using reward function, reward_model has no attribute 'modules'

  #2273 opened a month ago by j-murel
  1

• A problem with ppotrainer

  #2267 opened a month ago by serendipity800
  0

• Helper function for getting reward model and judge

  #2271 opened a month ago by qgallouedec
  0

• multi-gpu training

  #2256 opened a month ago by innat
  1

• trl/examples/scripts /sft_vlm.py evaluation

  #2254 opened a month ago by saxenarohit
  4

• Segmentation fault (core dumped) when running the sft scripts

  #2257 opened a month ago by SinclairCoder
  3

• `LogCompletionsCallback` can't find the tokenizer

  #2260 opened a month ago by qgallouedec
  0

• Use Offline Teacher Logits for Generalized Knowledge Distillation Trainer

  #2255 opened a month ago by mertege
  2

• Usage of `padding_free`

  #2242 opened a month ago by zwhe99
  2

• TypeError: XPOTrainer.training_step() takes 3 positional arguments but 4 were given

  #2247 opened a month ago by ArcherShirou
  2