onnx/models

Available INT4 ONNX models?

laumecha opened this issue · 4 comments

laumecha commented

Hello everyone,

Does anyone know where I can find INT4 ONNX quantized models with reasonable accuracy?

ramkrishna2910 commented

Hi @laumecha
Can you add some more details on what models you are specifically looking for?
You can find some int4 onnx models in the Huggingface here.

laumecha commented

Thank you very much for responding. I was looking for classification or object detection segmentation models.

ramkrishna2910 commented

We currently dont have any int4 models available. But you can use publicly available tools like the neural-compressor to generate int4 models.

ramkrishna2910 commented

Closing due to inactivity.