hijeffwu

Pai-Megatron-Patch

The official repo of Pai-Megatron-Patch for LLM & VLM large scale training developed by Alibaba Cloud.

Megatron-DeepSpeed

Ongoing research training transformer language models at scale, including: BERT & GPT-2

Yuan-2.0

Yuan 2.0 Large Language Model

cluster-health

DeepSpeed

DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.

Megatron-DeepSpeed

Ongoing research training transformer language models at scale, including: BERT & GPT-2

CoLLiE

Collaborative Training of Large Language Models in an Efficient Way

hijeffwu/Megatron-DeepSpeed

Ongoing research training transformer language models at scale, including: BERT & GPT-2