EleutherAI/oslo

TODO : switch code base of expert parallel from colossalai to deepspeed

scsc0511 opened this issue · 1 comments

scsc0511 commented

Describe a TODO feature

  • Existing code is based on colossalai but this code is not proper for multi parallelism.
  • Switch the code base of expert parallel to deepspeed

Assignees

-@scsc0511

hyunwoongko commented

I think this is duplicated with #4
how about closing one of those issues?