CarperAI/trlx

Issues

• OOM error with PEFT LoRA on Llama2-7B

  #601 opened 4 months ago by arpaiva
  1

• Load the checkpoint fails

  #600 opened 4 months ago by AfraAmini
  0

• Issue of tensors share memory

  #591 opened 9 months ago by heraldiclily
  2

• Multi-GPU training errors with peft

  #581 opened a year ago by AliengirlLiv
  1

• cannot import name 'flatten_dataclass' from 'trlx.data.ilql_types'

  #599 opened 5 months ago by AfraAmini
  0

• maybe bug in prepare & load's order

  #598 opened 5 months ago by daiwk
  1

• Error when running Ray Tune to launch hyperparameter sweep

  #597 opened 6 months ago by Jing-L97
  1

• Issue since most recent transformers update

  #580 opened a year ago by siddharthverma314
  1

• Crash when using save_state with deepspeed: `model.state_dict` functions incompatible with new deepspeed.

  #596 opened 6 months ago by JohannesAck
  0

• Data Loader Bug when running t5_summarization_daily_cnn.py

  #595 opened 7 months ago by yunanyan
  0

• Unable to load and run inference on finetuned Alpaca model

  #549 opened a year ago by doyled-it
  7

• Why train dataloader is not prepared by Accelerator

  #594 opened 8 months ago by Jiaxin-Wen
  0

• TRLX Environment customization

  #593 opened 9 months ago by heraldiclily
  0

• Unable to load the trained model to do the inference

  #545 opened a year ago by CSerxy
  9

• when i use trlx ppotrainer train a model llama 13b model, but saved huggingface mode ,but when it inference , it has some strange keys ,and the inference result did not show ,it also have no error , it seems the result disapper

  #584 opened a year ago by ldh127
  1

• Runtime error when running examples (ilql_sentiments_t5.py)

  #587 opened a year ago by youxiho1
  2

• [New Feature Request] Add KTO

  #590 opened a year ago by 1485840691-eng
  0

• RLHF text summarization diverges

  #589 opened a year ago by AlisonWen
  0

• Integration of Self-Play Fine-Tuning (SPIN) Method for Enhancing Large Language Models

  #588 opened a year ago by SeungyounShin
  0

• MPT is not working

  #585 opened a year ago by ouhenio
  0

• Attention mask when calculating log ratio for PPO

  #582 opened a year ago by kmy17518
  0

• multigpu support for summarization ppo example

  #571 opened a year ago by sayan1101
  3

• resume_from_checkpoint doesn't work

  #577 opened a year ago by andrewsiah
  1

• Support parallel reward_fn in PPO training

  #574 opened a year ago by Jingru
  0

• `position_ids` error in accelerate PPO trainer

  #564 opened a year ago by pbarragan
  3

• TypeError: reward_fn() got an unexpected keyword argument 'tokenizer'

  #569 opened a year ago by sayan1101
  1

• Question about saving peft checkpoint

  #565 opened a year ago by nhanph
  2

• Problem with LLama training with LoRA

  #567 opened a year ago by freQuensy23-coder
  3

• How to generate reward-labeled dataset

  #561 opened a year ago by mikkelmedm
  0

• How to train LLaMA2 on the summarize_rlhf example?

  #559 opened a year ago by missflash
  0

• Increasing max new tokens for generation arguments lead to errors

  #553 opened a year ago by wise-east
  3

• strange design

  #501 opened a year ago by efengx
  1

• Sanity check: SFT Model should be frozen (PPO)

  #517 opened a year ago by Apsod
  2

• Reward model negative numbers meaning

  #521 opened a year ago by GenVr
  2

• Model does not load in the expected dtype

  #535 opened a year ago by AugustasMacijauskas
  5

• RuntimeError: module must have its parameters and buffers on device

  #544 opened a year ago by Adaickalavan
  4

• Add support for Falcon 7B/40B

  #532 opened a year ago by cvetanovskaa
  1

• Memory occupy with multi GPUs Training

  #548 opened a year ago by yuanyaaa
  1

• Caught signal 7 (Bus error: nonexistent physical address)

  #536 opened a year ago by Adaickalavan
  5

• ILQL training batch2 tensor dimensions error

  #540 opened a year ago by GenVr
  2

• Direct Policy Optimization

  #504 opened 2 years ago by Reichenbachian
  4

• Add support for LLaMA2

  #533 opened a year ago by cvetanovskaa
  1

• Implement Asynchronous PPO

  #531 opened a year ago by Dahoas
  0

• ppo using GLM2-6b as a backbone?

  #523 opened a year ago by fanxinyun1991
  1

• support base model + multi adapter for actor, critic, ref and reward model

  #518 opened 2 years ago by akk-123
  0

• 8-bit inference

  #512 opened 2 years ago by glerzing
  0

• sanity check: PPO `log_ratio` should be zero when training is disabled

  #508 opened 2 years ago by TobiasNorlund
  1

• Add support for safetensors

  #505 opened 2 years ago by glerzing
  0

• About the weight of word embedding being nan

  #503 opened 2 years ago by ItGirls
  1

• Use tiny models for the tests

  #502 opened 2 years ago by glerzing
  0