/LargeScale

Primary LanguagePythonOtherNOASSERTION

LargeScale

Set DATA_PATH, MULTITASK_DATA_PATH, CHECKPOINT_PATH in configs/glm-130b/glm-130b.sh and HOST_FILE_PATH in scripts/submit_gpu.sh. Run the following scripts to reproduce GLM-130B's training.

bash scripts/submit_gpu.sh configs/glm-130b/glm-130b.sh

At least 24 DGX-A100 (40G) is needed to lanuch training. A more detailed README will be released soon.