Set DATA_PATH
, MULTITASK_DATA_PATH
, CHECKPOINT_PATH
in configs/glm-130b/glm-130b.sh
and HOST_FILE_PATH
in scripts/submit_gpu.sh
. Run the following scripts to reproduce GLM-130B's training.
bash scripts/submit_gpu.sh configs/glm-130b/glm-130b.sh
At least 24 DGX-A100 (40G) is needed to lanuch training. A more detailed README will be released soon.