Error trying Quantize 7B model to 8-bit
guoti777 opened this issue · 2 comments
guoti777 commented
when run :
python -m llama.llama_quant decapoda-research/llama-7b-hf c4 --wbits 8 --save pyllama-7B8b.pt
got error:
OSError: Unable to load weights from pytorch checkpoint file for '/home/jima/.cache/huggingface/hub/models--decapoda-research--llama-7b-hf/snapshots/5f98eefcc80e437ef68d457ad7bf167c2c6a1348/pytorch_model-00002-of-00033.bin' at '/home/jima/.cache/huggingface/hub/models--decapoda-research--llama-7b-hf/snapshots/5f98eefcc80e437ef68d457ad7bf167c2c6a1348/pytorch_model-00002-of-00033.bin'. If you tried to load a PyTorch model from a TF 2.0 checkpoint, please set from_tf=True.
wanweilove commented
uninstall tensorflow?
juncongmoo commented
Please uninstall tensorflow. The transformers lib will try to import TF even we don't need it.