Is there a typo here?
Closed this issue · 0 comments
digger-yu commented
with
\ColossalChat\examples\train_sft.sh
\ColossalChat\README.md line 125 etc.
torchrun --standalone --nproc_per_node=4 train_sft.py \
--pretrain "/path/to/LLaMa-7B/" \
--model 'llama' \
--strategy colossalai_zero2 \
--log_interval 10 \
--save_path /path/to/Coati-7B \
--dataset /path/to/data.json \
--batch_size 4 \
--accimulation_steps 8 \
--lr 2e-5 \
--max_datasets_size 512 \
--max_epochs 1 \
Are the accimulation_steps parameters here correct?