01-ai/Yi

是否支持在npu上微调和推理

yyuan312 opened this issue · 3 comments

yyuan312 commented

Reminder

  • I have searched the Github Discussion and issues and have not found anything similar to this.

What issue do you find in Yi docs?

缺乏在npu设备上微调和推理的代码

What is your suggestion?

增加兼容npu上微调和推理的代码

Any reference?

No response

Are you willing to submit a PR?

  • I'm willing to submit a PR!
Yimi81 commented

@yyuan312 该框架支持对Yi系列模型在npu上的微调和推理，请尝试

yyuan312 commented

我试过了，910B 推理报OOM ，需要分布式推理，但是没有分布式推理的脚本

Yimi81 commented

我记得是支持分布式推理的啊，脚本在readme里就有写，我没有npu没法测试非常抱歉，你可以在其仓库的issues里找一下？