kssteven418/I-BERT

Bugs in the code

Opened this issue · 0 comments

hsiehjackson commented

🐛 Bug

  1. x_scaling_factor in transformer_sentence_encoder.py:
    I think the code here should modified scaling_factor to x_scaling_factor. Or do we use the same scaling factor for all transformer encoder layers?
  2. freeze_model for IntSoftmax:
    I think we should implement fix function in IntSoftmax because we have QuantAct here. If we don't implement the fix function, we will skip fixing the QuantAct in IntSoftmax from here.

Problem

I am recently trying to implement IBERT on TVM an I found I should add FixedPointMul and SymmetricQuantFunction operators in TVM. Do you have any existing implementation? Thanks!