spcl/QuaRot

How to get a fake quantized model?

Closed this issue · 1 comments

mxjmtxrm commented

As title. How to get a HF model whose state dict key is same as the origin model, and the weights are fake quantized.

sashkboos commented

Thanks @mxjmtxrm for your issue.

We are working on this and will publish the quantized models soon.