X-PLUG/mPLUG-Owl

unable to load model

segalinc opened this issue · 3 comments

segalinc commented

Hello,
I am trying to load the model using the demo but I get

    tokenizer, model, image_processor, context_len = load_pretrained_model(model_path, None, model_name, load_8bit=False, load_4bit=False, device="cuda")
  File "mPLUG-Owl/mPLUG-Owl2/mplug_owl2/model/builder.py", line 110, in load_pretrained_model
    vision_tower.to(device=device, dtype=torch.float16)
  File "venv/lib/python3.10/site-packages/transformers/modeling_utils.py", line 1900, in to
    return super().to(*args, **kwargs)
  File "venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1145, in to
    return self._apply(convert)
  File "venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 797, in _apply
    module._apply(fn)
  File "venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 797, in _apply
    module._apply(fn)
  File "venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 820, in _apply
    param_applied = fn(param)
  File "venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1143, in convert
    return t.to(device, dtype if t.is_floating_point() or t.is_complex() else None, non_blocking)
NotImplementedError: Cannot copy out of meta tensor; no data!
LianghuiGuo commented

Same problem! I met this when using web ui

LianghuiGuo commented

Hello, I am trying to load the model using the demo but I get

    tokenizer, model, image_processor, context_len = load_pretrained_model(model_path, None, model_name, load_8bit=False, load_4bit=False, device="cuda")
  File "mPLUG-Owl/mPLUG-Owl2/mplug_owl2/model/builder.py", line 110, in load_pretrained_model
    vision_tower.to(device=device, dtype=torch.float16)
  File "venv/lib/python3.10/site-packages/transformers/modeling_utils.py", line 1900, in to
    return super().to(*args, **kwargs)
  File "venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1145, in to
    return self._apply(convert)
  File "venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 797, in _apply
    module._apply(fn)
  File "venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 797, in _apply
    module._apply(fn)
  File "venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 820, in _apply
    param_applied = fn(param)
  File "venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1143, in convert
    return t.to(device, dtype if t.is_floating_point() or t.is_complex() else None, non_blocking)
NotImplementedError: Cannot copy out of meta tensor; no data!

Did you solve it? Thanks

Tianchong-Jiang commented

I solved this problem by using a GPU with larger RAM.
This thread suggest that "This error is caused by Accelerate auto-offloading weights to either the CPU or disk because of insufficient memory on the GPU" so having sufficient GPU RAM solves it.

  • 👍1