/sequence-likelihood-calibration

Reproduction of SLiC-HF: Sequence Likelihood Calibration with Human Feedback

Primary LanguagePython

SLIC-HF

Reproduction of SLiC-HF: Sequence Likelihood Calibration with Human Feedback

(Work in progress)

Installation

# Install conda
wget https://repo.anaconda.com/miniconda/Miniconda3-py39_23.3.1-0-Linux-x86_64.sh
sh Miniconda3-py39_23.3.1-0-Linux-x86_64.sh

# Install this
source devops/install.sh

Running

# Local development
sh src/do_everything_debug.sh

# Full training
sh src/do_everything_large.sh

Tensorboard

  • Long short experiment - Comparing src/train/train_slic.py::slic_loss function vs src/train/train_slic.py::slic_loss_logits function on t5-base.