JulesGM/Marg-Li-CoT

Jupyter Notebook

Using Reinforcement Learning to Guide Chains of Thought

Uses Hugging Face TRL for PPO
Uses Hugging Face Peft for LoRA.
Uses Bitsandbytes internally for 4bits and 8bits reference model modes.
Uses our QLora standalone lib for QLora.

With TRL:

Where the reinforcement learning is located.

Approach SFT:

There, one finds the supervised baselines:

Generate, then learn, masked.