Cornell-RelaxML/QuIP

will it support group quant

oreo0906 opened this issue · 1 comments

I confused the method ldlq_Rg dont support group quantization.

we did not investigate group quantization in this codebase, which is why it is not supported. but the method is compatible with group quantization. but incoherence and grouping aim to do similar things. Incoherence reduces overall variation in the weights via a random orthogonal matmul, while grouping reduces the effect of variation in the weights when quantizing by adding additional scaling params.