Is the Table.3 accuracy tested with dequantized weights, or tested on real accelerated quantized kernels?

[vovoluck]

For the zero-shot accuracy above, is the accuracy tested on dequantized weights, or tested on real accelerated quantized kernels?

If it is tested with dequantized weights, is there any accuracy difference between dequantized weights and real hardware quantized operations? Have you validated this?
@synxlin

[ys-2020]

Hi @vovoluck, thanks for your interests in QServe and QoQ.

The accuracy is tested with dequantized weights rather than the real accelerated kernels. We have verified that there is NO accuracy difference between dequantized weights and real hardware quantized operations.