Error using xcomposer: Can't move a model that's been offloaded
gonzalu opened this issue · 1 comments
gonzalu commented
I am getting this error:
File "E:\ComfyUI\python_embeded\Lib\site-packages\accelerate\big_modeling.py", line 428, in wrapper
raise RuntimeError("You can't move a model that has some modules offloaded to cpu or disk.")
RuntimeError: You can't move a model that has some modules offloaded to cpu or disk.
Full error here:
Fetching 19 files: 100%|██████████████████████████████████████████████████████████████████████| 19/19 [00:07<00:00, 2.54it/s]
Set max length to 1024
We will use 90% of the memory on device 0 for storing the model, and 10% for the buffer to avoid OOM. You can set `max_memory` in to a higher value to use more memory (at your own risk).
Loading checkpoint shards: 100%|████████████████████████████████████████████████████████████████| 2/2 [00:08<00:00, 4.23s/it]
Some parameters are on the meta device device because they were offloaded to the cpu.
You shouldn't move a model when it is dispatched on multiple devices.
!!! Exception during processing !!!
Traceback (most recent call last):
File "E:\ComfyUI\ComfyUI\execution.py", line 151, in recursive_execute
output_data, output_ui = get_output_data(obj, input_data_all)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "E:\ComfyUI\ComfyUI\execution.py", line 81, in get_output_data
return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "E:\ComfyUI\ComfyUI\execution.py", line 74, in map_node_over_list
results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "E:\ComfyUI\ComfyUI\custom_nodes\Comfyui_image2prompt\src\image2text.py", line 42, in get_model
return (InternlmVLModle(device=device,low_memory=low_memory),)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "E:\ComfyUI\ComfyUI\custom_nodes\Comfyui_image2prompt\src\internlm_model.py", line 31, in __init__
self.model_path, self.model, self.tokenizer = self.load_model(device=device, dtype=dtype)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "E:\ComfyUI\ComfyUI\custom_nodes\Comfyui_image2prompt\src\internlm_model.py", line 96, in load_model
).to(device).eval()
^^^^^^^^^^
File "E:\ComfyUI\python_embeded\Lib\site-packages\accelerate\big_modeling.py", line 428, in wrapper
raise RuntimeError("You can't move a model that has some modules offloaded to cpu or disk.")
RuntimeError: You can't move a model that has some modules offloaded to cpu or disk.
whisky-12 commented
Hello, may I ask if this issue has been resolved? We have also encountered similar problems and hope to obtain a solution