intersun/PKD-for-BERT-Model-Compression

Issues

• 论文公式3是交叉熵公式，但为什么代码用的KL散度来实现的公式3？

  #14 opened 2 years ago by jinxinglu
  0

• 请问一个问题

  #10 opened 5 years ago by Smile-shirley
  5

• Why do you set for KD.Full like this [fix_pooler=True]?

  #13 opened 4 years ago by GeondoPark
  0

• Some questions about layer number (model size)

  #12 opened 4 years ago by ZLKong
  0

• Not able to reproduce results

  #11 opened 5 years ago by ashim95
  0

• Reproducing results

  #9 opened 5 years ago by pawankmrs
  3

• Trying to do distillation for regression task

  #8 opened 5 years ago by smr97
  1

• How do I run student predictions?

  #7 opened 5 years ago by smr97
  2

• RuntimeError: Function AddBackward0 returned an invalid gradient at index 1 - expected type torch.cuda.HalfTensor but got torch.cuda.FloatTensor

  #6 opened 5 years ago by yg33717
  2

• question on pretrained/bert_config.json

  #3 opened 5 years ago by Seohyeong
  1

• Result is different...

  #5 opened 5 years ago by jdh3577
  2

• What's the version of python and pytorch of this project?

  #4 opened 5 years ago by vikotse
  1

• Running NLI_KD_training.py raises the following error.

  #1 opened 5 years ago by non-swimmer
  2

• Where to download the pretrained weights?

  #2 opened 5 years ago by thudzj
  1