How to use default n bit in QAT ?
Opened this issue · 3 comments
System information
- TensorFlow version (you are using):2.5
- Are you willing to contribute it (Yes/No):Yes
Motivation
What is the use case and how does it broadly benefits users? Prioritization and whether a feature is added is based on how it
helps the community and the feature's maintenance costs.
As examples:
-
Instead of, "Enable the technique for my model," "Enable this technique to work better with standard object detection models, including R-CNN (link) and SSD (link)" is stronger.
-
Instead of, "Try something more customized with the technique," "Implement a variant of the algorithm described in equations X and Y of this paper" is clearer.
Describe the feature
sample code to QAT with 4 bit or n bit
Describe how the feature helps achieve the use case
Describe how existing APIs don't satisfy your use case (optional if obvious)
As examples:
-
You tried using APIs X and Y and were able to do Z. However, that was not sufficient because of ...
-
You achieved your use case with the code snippet W. However, this was more difficult than it should be because of ... (e.g. ran into issue X or had
to do Y).
anyone can help me？
We have a experimental n-bit quantization scheme here: https://github.com/tensorflow/model-optimization/tree/master/tensorflow_model_optimization/python/core/quantization/keras/experimental/default_n_bit
We have a plan to release an example that shows how to use this default_n_bit scheme on tensorflow official vision model. (https://github.com/tensorflow/models/tree/master/official/vision) but it's WIP. Please wait a bit.
Note that: Quantization scheme is depends on the platform you'd like to deploy. (e.g. TFLite has fused conv-relu op, so we don't add fakequant between conv-relu on default 8 bit TFLite scheme.) The n-bit experimental scheme is an extension of the TFLite 8bit scheme.
Hello, any update on the default n bit quantization? Is a tutorial available by any chance?