Gourieff/comfyui-reactor-node

AMD GPU | ReActor face_restore_model not working

TobiasKlein2004 opened this issue · 5 comments

TobiasKlein2004 commented

First, confirm

  • I have read the instruction carefully
  • I have searched the existing issues
  • I have updated the extension to the latest version

What happened?

I had this working setup:
image

Now I want to remove the blur from her face by utilizing a face restore model like codeformer or GFPGAN.
image

Both of them result in this Error and I have no Idea why:

Error occurred when executing ReActorFaceSwap:

Input type (torch.FloatTensor) and weight type (PrivateUse1FloatType) should be the same or input should be a MKLDNN tensor and weight is a dense tensor

File "C:\Users\klein\stablediffusion\comfyui_amd_gpu\comfyui\execution.py", line 154, in recursive_execute
output_data, output_ui = get_output_data(obj, input_data_all)
File "C:\Users\klein\stablediffusion\comfyui_amd_gpu\comfyui\execution.py", line 84, in get_output_data
return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
File "C:\Users\klein\stablediffusion\comfyui_amd_gpu\comfyui\execution.py", line 77, in map_node_over_list
results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
File "C:\Users\klein\stablediffusion\comfyui_amd_gpu\ComfyUI\custom_nodes\comfyui-reactor-node\nodes.py", line 181, in execute
self.face_helper.get_face_landmarks_5(only_center_face=False, resize=640, eye_dist_threshold=5)
File "C:\Users\klein\stablediffusion\comfyui_amd_gpu\ComfyUI\custom_nodes\comfyui-reactor-node\r_facelib\utils\face_restoration_helper.py", line 151, in get_face_landmarks_5
bboxes = self.face_det.detect_faces(input_img)
File "C:\Users\klein\stablediffusion\comfyui_amd_gpu\ComfyUI\custom_nodes\comfyui-reactor-node\r_facelib\detection\retinaface\retinaface.py", line 217, in detect_faces
loc, conf, landmarks, priors = self.__detect_faces(image)
File "C:\Users\klein\stablediffusion\comfyui_amd_gpu\ComfyUI\custom_nodes\comfyui-reactor-node\r_facelib\detection\retinaface\retinaface.py", line 164, in __detect_faces
loc, conf, landmarks = self(inputs)
File "C:\Users\klein\stablediffusion\comfyui_amd_gpu\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
return forward_call(*args, **kwargs)
File "C:\Users\klein\stablediffusion\comfyui_amd_gpu\ComfyUI\custom_nodes\comfyui-reactor-node\r_facelib\detection\retinaface\retinaface.py", line 129, in forward
out = self.body(inputs)
File "C:\Users\klein\stablediffusion\comfyui_amd_gpu\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
return forward_call(*args, **kwargs)
File "C:\Users\klein\stablediffusion\comfyui_amd_gpu\lib\site-packages\torchvision\models\_utils.py", line 69, in forward
x = module(x)
File "C:\Users\klein\stablediffusion\comfyui_amd_gpu\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
return forward_call(*args, **kwargs)
File "C:\Users\klein\stablediffusion\comfyui_amd_gpu\lib\site-packages\torch\nn\modules\conv.py", line 463, in forward
return self._conv_forward(input, self.weight, self.bias)
File "C:\Users\klein\stablediffusion\comfyui_amd_gpu\lib\site-packages\torch\nn\modules\conv.py", line 459, in _conv_forward
return F.conv2d(input, weight, bias, self.stride,

Can someone please help me 😅

Steps to reproduce the problem

just as described above

Sysinfo

Windows 11
Chrome
AMD RX6600 using directml

Relevant console log

ERROR:root:!!! Exception during processing !!!
ERROR:root:Traceback (most recent call last):
  File "C:\Users\klein\stablediffusion\comfyui_amd_gpu\comfyui\execution.py", line 154, in recursive_execute
    output_data, output_ui = get_output_data(obj, input_data_all)
  File "C:\Users\klein\stablediffusion\comfyui_amd_gpu\comfyui\execution.py", line 84, in get_output_data
    return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
  File "C:\Users\klein\stablediffusion\comfyui_amd_gpu\comfyui\execution.py", line 77, in map_node_over_list
    results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
  File "C:\Users\klein\stablediffusion\comfyui_amd_gpu\ComfyUI\custom_nodes\comfyui-reactor-node\nodes.py", line 181, in execute
    self.face_helper.get_face_landmarks_5(only_center_face=False, resize=640, eye_dist_threshold=5)
  File "C:\Users\klein\stablediffusion\comfyui_amd_gpu\ComfyUI\custom_nodes\comfyui-reactor-node\r_facelib\utils\face_restoration_helper.py", line 151, in get_face_landmarks_5
    bboxes = self.face_det.detect_faces(input_img)
  File "C:\Users\klein\stablediffusion\comfyui_amd_gpu\ComfyUI\custom_nodes\comfyui-reactor-node\r_facelib\detection\retinaface\retinaface.py", line 217, in detect_faces
    loc, conf, landmarks, priors = self.__detect_faces(image)
  File "C:\Users\klein\stablediffusion\comfyui_amd_gpu\ComfyUI\custom_nodes\comfyui-reactor-node\r_facelib\detection\retinaface\retinaface.py", line 164, in __detect_faces
    loc, conf, landmarks = self(inputs)
  File "C:\Users\klein\stablediffusion\comfyui_amd_gpu\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "C:\Users\klein\stablediffusion\comfyui_amd_gpu\ComfyUI\custom_nodes\comfyui-reactor-node\r_facelib\detection\retinaface\retinaface.py", line 129, in forward
    out = self.body(inputs)
  File "C:\Users\klein\stablediffusion\comfyui_amd_gpu\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "C:\Users\klein\stablediffusion\comfyui_amd_gpu\lib\site-packages\torchvision\models\_utils.py", line 69, in forward
    x = module(x)
  File "C:\Users\klein\stablediffusion\comfyui_amd_gpu\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "C:\Users\klein\stablediffusion\comfyui_amd_gpu\lib\site-packages\torch\nn\modules\conv.py", line 463, in forward
    return self._conv_forward(input, self.weight, self.bias)
  File "C:\Users\klein\stablediffusion\comfyui_amd_gpu\lib\site-packages\torch\nn\modules\conv.py", line 459, in _conv_forward
    return F.conv2d(input, weight, bias, self.stride,
RuntimeError: Input type (torch.FloatTensor) and weight type (PrivateUse1FloatType) should be the same or input should be a MKLDNN tensor and weight is a dense tensor

Additional information

No response

Gourieff commented

Unfortunately I don't have AMD GPU to test this point, but you can read the topics below, perhaps some advices will help you to resolve the issue:

#76

#40 (comment)

Shadow-MasterTf commented

I have the exact same issue as @TobiasKlein2004 mentioned.

Running Windows 11, RX 7900XTX using directml.

It runs fine until I set the "face_restore_model" to anything other then "none" I have tried both codeformer or GFPGAN and get the same error message.

RuntimeError: Input type (torch.FloatTensor) and weight type (PrivateUse1FloatType) should be the same or input should be a MKLDNN tensor and weight is a dense tensor

patientx commented

UPDATE :
first of all
"device = torch.device('dml')
doesn't even work here (AMD) because the device name is not "DML" but "privateuseone" with torchdirectml on windows
and second ;
I tried just commenting out the others

#device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')

if torch.cuda.is_available():

device = torch.device('cuda')

elif torch.backends.mps.is_available():

device = torch.device('mps')

elif hasattr(torch,'dml'):

device = torch.device('dml')

else:

device = torch.device('cpu')

just left the last line, still got errors. Finally I deleted all those lines and just used EITHER OF THESE

device = torch.device('cpu')

OR

device = torch.device('privateuseone')

Both of them works, and it doesn't seem like there is much speed difference here either.

Still DON'T KNOW > if this is a one time thing or the final solution

but at least I am sure DML device name doesn't work , here is what comfyui log says if I left it at ::

device = torch.device('dml')

RuntimeError: Expected one of cpu, cuda, ipu, xpu, mkldnn, opengl, opencl, ideep, hip, ve, fpga, ort, xla, lazy, vulkan, mps, meta, hpu, mtia, privateuseone device type at start of device string: dml

I am the author of one of the mentioned topics that has been marked solved , at the time I was able to overcome this by deleting torch and torchvision from the requirements.txt so the extension keeps using the default installed ones with comfyui (this is no longer viable because they are removed from the txt now) BUT even after that, this problem keeps happening from time to time. Sometimes it requires a reboot sometimes just changing to a different workflow or setting up a workflow from stratch.

There is a similar problem with controlnet extension Mikubill/sd-webui-controlnet#901 (comment) the controlnet dev seems to have solved it, maybe check that solution @Gourieff ?

Gourieff commented

🤔 Ok, I'll take a look... Thanks for your research!
What a strange logic to call the "dml" device as "privateuseone"...

patientx commented

just make amd use cpu if it is easier so far I have not seen any speed changes maybe if we were doing work on a lot of faces back to back like the ones in facefusion standalone this would make sense but here more or less the same speed. (I do three images with batch encoding the resulting speed is the same with both cpu & "privateuseone" , maybe it is not activated correctly but let me tell you from the times this worked before I never seen it work faster so probably we were always using cpu)

  • 👍1