Porting GPTQ to CPU?
yiliu30 opened this issue · 2 comments
yiliu30 commented
Is it possible to run GPTQ on a machine that has only CPUs? If not, is there a plan for it?
aljungberg commented
You can use a GPTQ quantized model with llama.cpp by using this conversion script I believe.
Hiwyl commented
just quant model on CPU?