How to use mscclpp as a backend in pytorch
wangfakang opened this issue · 4 comments
How to use mscclpp as a backend in pytorch? thx.
We don't support this right now. mscclpp not compatible with nccl interface and have no plan for this.
@Binyang2014 thanks. Another question, What are the key points for improving mscclpp compared to NCCL busbw?
As we motioned in README. 1. We provide light-weight API, which can be used to implement communication collectives. NCCL provides general interfaces which cannot be customized based on network topo and hardware spec. 2. Our API is 1-sided 0-copy, which means we don't introduce temp buffers. Also 1-side communication gives us the ability to overlap some ops. 3. NCCL only uses SM copy. We introduce DMA copy with proxy channel. We can choose the best communication methods based on use-case.
@wangfakang BTW, I think this PR #319 will support partial NCCL API. Pls follow the readme https://github.com/microsoft/mscclpp/blob/main/apps/nccl/README.md if you would like to try