deepseek-ai/DeepSeek-MoE

Issues

• Ablation studies for DeepSeekMoE

  #42 opened a month ago by Psycoy
  0

• 用来训练deepseek-v2-coder-lite-instruction会挂起

  #41 opened 4 months ago by bao-xiaoyi
  0

• 您们会开源DeepSeekMoE 2B模型吗?

  #16 opened a year ago by win10ogod
  6

• 训练MoE的时候会出现loss = 0 的情况

  #39 opened 5 months ago by AlenjandroWang
  0

• 专家并行是怎么配置的？ 有配置代码吗

  #38 opened 5 months ago by ninglonglong
  0

• deepseek-moe模型在进行lora微调训练时loss值会突然变为0一直到最后，导致推理异常。

  #27 opened 10 months ago by hangchen426926
  3

• Finetune with deepspeed: type mismatch

  #35 opened 7 months ago by YeZiyi1998
  3

• why <|EOT|> ?

  #37 opened 6 months ago by BING-LLL
  0

• Close expert parallel in vllm

  #36 opened 7 months ago by trebladev
  0

• 单卡A100-80G推理速度慢

  #34 opened 8 months ago by Dreaming-world
  0

• About expert capacities: Is there token-dropping during training?

  #29 opened 9 months ago by Spico197
  3

• MOE 并行怎么实现的？

  #31 opened 10 months ago by YunxinLi
  1

• 模型结果测评复现

  #30 opened 10 months ago by JustQJ
  1

• 您好，能否提供量化的方案

  #21 opened a year ago by edisonzf2020
  2

• No need to add epsilon 1e-20 in topk norm?

  #33 opened 10 months ago by MARD1NO
  0

• 能添加modelscope链接吗，这样可以更方便一些不能连hg的情况

  #32 opened 10 months ago by lll143653
  0

• 您们好请问准备开源的moe-145b什么时候准备上传呢?

  #25 opened a year ago by win10ogod
  3

• finetune后的模型输出异常

  #28 opened 10 months ago by JustQJ
  4

• load erros

  #24 opened 10 months ago by cooper12121
  2

• 请问现在支持在NPU设备上进行微调吗

  #26 opened 10 months ago by Tyx-main
  1

• Can you provide the inference version of DeepSeek based on vllm, deepspeed and tensorrt-llm

  #23 opened a year ago by Eutenacity
  1

• How to fully finetune MoE on multiple nodes

  #12 opened a year ago by ftgreat
  1

• 您们有计划支持llama.cpp这个项目吗

  #15 opened a year ago by hqu-little-boy
  1

• 您们能够开源复现模型架构的训练项目吗?

  #7 opened a year ago by win10ogod
  3

• 关于flash_attn

  #20 opened a year ago by GXKIM
  1

• 非常棒的工作，有没有微信沟通群呢

  #22 opened a year ago by dawson-chen
  1

• Will it compare performance with llama-moe?

  #11 opened a year ago by ccccj
  1

• Selective precision In gate and norm may conflict with deepspeed？

  #18 opened a year ago by drxmy
  1

• GPU utils is low compared with dense model

  #5 opened a year ago by charliedream1
  4

• #feature request# DeepSeek-Moe for code

  #8 opened a year ago by Xingxiangrui
  1

• Question about AddAuxiliaryLoss?

  #17 opened a year ago by KaiWU5
  1

• deepseek-moe-16b inference speed is slower than Baichuan-13b

  #13 opened a year ago by ifromeast
  3

• 开源的MoE模型支持中文吗？

  #6 opened a year ago by uloveqian2021
  4

• inference tools like vllm can support?

  #2 opened a year ago by zhang001122
  3

• flash atten

  #19 opened a year ago by GXKIM
  0

• 求助：模型无法加载

  #3 opened a year ago by KMnO4-zx
  4

• The released DeepSeekMoE 16B Base has 3 different vocab size

  #14 opened a year ago by drxmy
  2

• finetune 过程出错

  #10 opened a year ago by ifromeast
  1

• CUDA error: device-side assert triggered when trying to run the model

  #4 opened a year ago by intervitens
  2