will it support group quant
oreo0906 opened this issue · 1 comments
oreo0906 commented
I confused the method ldlq_Rg dont support group quantization.
jerry-chee commented
we did not investigate group quantization in this codebase, which is why it is not supported. but the method is compatible with group quantization. but incoherence and grouping aim to do similar things. Incoherence reduces overall variation in the weights via a random orthogonal matmul, while grouping reduces the effect of variation in the weights when quantizing by adding additional scaling params.