Does Megatron has plan to support llama pre-train？
Opened this issue · 2 comments
wen020 commented
Does Megatron has plan to support llama pre-train？
ethanhe42 commented
it's available through nemo which uses megatron https://github.com/NVIDIA/NeMo/blob/main/examples/nlp/language_modeling/conf/megatron_llama_config.yaml