How to Quantize CNN Layers (non MLP layers in general) using qserve?

[satabios]

Any thoughts or suggestions would be appreciated.

[kentang-mit]

Hi @satabios,

Thanks for your interest! Although CNN layer quantization has not been supported yet in QServe and LMQuant, we believe that the principles mentioned in the paper are still applicable (e.g. progressive quantization).

Best,
Haotian