mit-han-lab/distrifuser

Using video cards with different architectures

DmitryVN opened this issue · 1 comments

Can I use it for a tesla p40 and 3060 ti graphics card? (or 30-s with 40-s series)
thanks

It depends on your communication bandwidth between GPUs. If it is not large enough, the communication cannot be hidden in the computation, which will slow down the inference. Typically, an NVLink is essential. However, it is quite accessible nowadays. You can find it here.