alibaba/Pai-Megatron-Patch

Issues

• 有适配qwen2-vl的打算吗？

  #339 opened 4 months ago by divisionblur
  2

• 建议对deepseek-v2-coder-lite进行sft测试

  #342 opened 4 months ago by bao-xiaoyi
  5

• AssertionError: Rank 11: found NaN in local grad norm in backward pass before data-parallel communication collective. Device: 3

  #366 opened 3 months ago by lanfengmo
  0

• Possible bug in Mistral MCore <->HF Model conversions because of _extra_state layers

  #363 opened 3 months ago by abgoswam
  0

• llava run error

  #330 opened 5 months ago by yangzhipeng1108
  3

• 关于LLAMA 3.1模型的适配问题

  #361 opened 3 months ago by echo-valor
  1

• 对qwen-2.5扩充词表后loss飙升

  #360 opened 3 months ago by QianguoS
  0

• cannot import name 'TEDotProductAttentionMLA' when running `examples/deepseek_v2/run_mcore_deepseek.sh`

  #359 opened 3 months ago by dreasysnail
  1

• No module named 'megatron'

  #357 opened 4 months ago by yuanzhiyong1999
  0

• 打扰了，提个关于多机训练的issues

  #307 opened 5 months ago by CallmeZhangChenchen
  4

• llama3.1-8b转换megatron-mcore格式后模型大小从15G变成了71G，精度仍然为bf16，这是正常的吗

  #356 opened 4 months ago by kkkeepgoing
  2

• DeepSeek Vocab-size Mismatch

  #338 opened 4 months ago by Jiayi-Pan
  1

• 加入群聊失败， 第二个群也不能扫码加入了

  #351 opened 4 months ago by GeorgeSen
  7

• qwen2.5转换脚本转换时报错

  #354 opened 4 months ago by enze5088
  1

• 群满了

  #310 opened 4 months ago by zgf1005
  3

• [[: not found Zarr-based strategies will not be registered because of missing packages Traceback (most recent call last)

  #346 opened 4 months ago by aJupyter
  1

• optimizer offload

  #352 opened 4 months ago by leo-ztjht
  1

• 在转换模型的时候就报了一些bug

  #350 opened 4 months ago by Yanhong-Li
  0

• llama3.1 8b训练32k的上下文模型，训练时间长、并且loss偏大

  #348 opened 4 months ago by ARQlalala
  0

• llama3.1支持多数据集混合预训练

  #347 opened 4 months ago by Bob199511
  1

• 有适配minicpm的打算吗？

  #345 opened 4 months ago by adol001
  0

• llama7b OOM问题

  #343 opened 4 months ago by mxjmtxrm
  2

• qwen2-sft 训练起步阶段就卡住

  #325 opened 5 months ago by baisechundu
  4

• 关于llava适配的问题

  #333 opened 4 months ago by divisionblur
  2

• AssertionError: First dimension of the tensor should be divisible by tensor parallel size

  #332 opened 4 months ago by pizts
  0

• deepseek模型转换问题

  #327 opened 5 months ago by bao-xiaoyi
  7

• TypeError: get_cpu_offload_context() missing 1 required positional argument: 'weight_offloading'

  #324 opened 5 months ago by ben-8878
  2

• 关于使用idxmap格式finetune qwen2

  #319 opened 5 months ago by Gloid59
  2

• Qwen2 0.5B 和 1.5B的模型是否应该将这个参数去掉?

  #296 opened 5 months ago by MrWaterZhou
  1

• OSError: [Errno 28] No space left on device 请教

  #302 opened 5 months ago by shyzzz521
  2

• Mcore是不支持pp吗？

  #312 opened 5 months ago by divisionblur
  3

• starcoder依赖哪个版本的megatron-lm？

  #314 opened 5 months ago by bao-xiaoyi
  3

• Channel Loss支持

  #316 opened 5 months ago by echo-valor
  1

• 断点续训问题

  #318 opened 5 months ago by divisionblur
  1

• mmap数据格式问题

  #320 opened 5 months ago by bao-xiaoyi
  1

• 安装pyarrow失败

  #321 opened 5 months ago by xiaoquanWu
  1

• mcore 权重转换不支持pp>1

  #322 opened 5 months ago by xs1997zju
  2

• 使用flash-attn训练Qwen1.5 1.8B 加速效果不明显

  #323 opened 5 months ago by coder-wangzhen
  1

• [rank2]: AttributeError: 'IndexedDataset' object has no attribute 'sizes'

  #293 opened 6 months ago by wccccp
  1

• QwenTokenizer与Qwen2Tokenizer

  #295 opened 6 months ago by sexan
  3

• 保存的checkpoints中缺少distrib_optim.pt

  #315 opened 5 months ago by shizikachen
  0

• 钉钉群满了

  #304 opened 5 months ago by divisionblur
  5

• seq len开大时，初始loss不正常

  #300 opened 5 months ago by Jayce1kk
  3

• 是否支持sharegpt格式数据？或者带"history"字段的多轮对话数据？

  #306 opened 6 months ago by jiejie1993
  1

• Flash-Attn 3的支持

  #308 opened 5 months ago by echo-valor
  1

• optimizer offloading 太强了

  #311 opened 5 months ago by 154912369
  1

• Missing key(s) in state_dict llama3 mcore转换后权重不匹配

  #303 opened 6 months ago by wuduher
  5

• bigcode-evaluation-harness 这个仓库应该是没有了

  #301 opened 6 months ago by CallmeZhangChenchen
  2

• [rank31]: OSError: error stat()ing file 数据集map问题

  #305 opened 6 months ago by shyzzz521
  0

• nvcr.io/nvidia/pytorch:23.12-py3镜像包冲突

  #294 opened 6 months ago by wuduher
  1