多卡跑baichuan2报错
uloveqian2021 opened this issue · 1 comments
uloveqian2021 commented
您好，多卡跑baichuan2报错
RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cuda:1!
lyhue1991 commented
多卡要用deepspeed，参考一下这个范例：https://github.com/xxm1668/chatglm2_lora/blob/main/train2.py