Error Message: `checkpoint` should be the path to a file containing a whole state dict
Closed this issue · 1 comments
dynamiccreator commented
Hello, have installed everything so far but got this when using command python server.py:
Loading Model ...
╭───────────────────── Traceback (most recent call last) ──────────────────────╮
│ /media/user-name/nvme-ext/deep-learning/alpaca-AI/Lora-training/alpaca_lora_4b │
│ it/text-generation-webui/server.py:235 in <module> │
│ │
│ 232 │ │ i = int(input())-1 │
│ 233 │ │ print() │
│ 234 │ shared.model_name = available_models[i] │
│ ❱ 235 shared.model, shared.tokenizer = load_model(shared.model_name) │
│ 236 if shared.args.lora: │
│ 237 │ add_lora_to_model(shared.args.lora) │
│ 238 │
│ │
│ /media/user-name/nvme-ext/deep-learning/alpaca-AI/Lora-training/alpaca_lora_4b │
│ it/text-generation-webui/custom_monkey_patch.py:21 in load_model_llama │
│ │
│ 18 │ print("Loading {} ...".format(model_path)) │
│ 19 │ t0 = time.time() │
│ 20 │ │
│ ❱ 21 │ model, tokenizer = load_llama_model_4bit_low_ram(config_path, model │
│ 22 │ │
│ 23 │ model = PeftModel.from_pretrained(model, lora_path, device_map={'': │
│ 24 │ print('{} Lora Applied.'.format(lora_path)) │
│ │
│ /media/user-name/nvme-ext/deep-learning/alpaca-AI/Lora-training/alpaca_lora_4b │
│ it/text-generation-webui/../repository/GPTQ-for-LLaMa/autograd_4bit.py:222 │
│ in load_llama_model_4bit_low_ram │
│ │
│ 219 │ │ │ if name in layers: │
│ 220 │ │ │ │ del layers[name] │
│ 221 │ │ make_quant_for_4bit_autograd(model, layers) │
│ ❱ 222 │ model = accelerate.load_checkpoint_and_dispatch( │
│ 223 │ │ model=model, │
│ 224 │ │ checkpoint=model_path, │
│ 225 │ │ device_map=device_map, │
│ │
│ /home/user-name/miniconda3/lib/python3.10/site-packages/accelerate/big_modelin │
│ g.py:479 in load_checkpoint_and_dispatch │
│ │
│ 476 │ │ ) │
│ 477 │ if offload_state_dict is None and "disk" in device_map.values(): │
│ 478 │ │ offload_state_dict = True │
│ ❱ 479 │ load_checkpoint_in_model( │
│ 480 │ │ model, │
│ 481 │ │ checkpoint, │
│ 482 │ │ device_map=device_map, │
│ │
│ /home/user-name/miniconda3/lib/python3.10/site-packages/accelerate/utils/model │
│ ing.py:815 in load_checkpoint_in_model │
│ │
│ 812 │ │ else: │
│ 813 │ │ │ raise ValueError(f"{checkpoint} containing more than one ` │
│ 814 │ else: │
│ ❱ 815 │ │ raise ValueError( │
│ 816 │ │ │ "`checkpoint` should be the path to a file containing a wh │
│ 817 │ │ │ f"checkpoint, or a folder containing a sharded checkpoint, │
│ 818 │ │ )
ValueError: `checkpoint` should be the path to a file containing a whole state
dict, or the index of a sharded checkpoint, or a folder containing a sharded
checkpoint, but got ./llama-13b-4bit.pt.
Any idea what's wrong?
dynamiccreator commented
Ah ,sorry it seems that I just had to move the .pt file myself to the same location where llama-13b-4bit folder is.