How to run it with llama-7b-hf-int4?
ZhUyU1997 opened this issue · 1 comments
ZhUyU1997 commented
I don't have enough GPU memory. Could you give the guide to run with INT4?
deep-diver commented
currently INT4 is not supported.
ZhUyU1997 opened this issue · 1 comments
I don't have enough GPU memory. Could you give the guide to run with INT4?
currently INT4 is not supported.