/kd_early_ranker

Covers a couple of approaches to training an early ranker with knowledge distillation from final ranker

Primary LanguagePythonGNU General Public License v3.0GPL-3.0

kd_early_ranker

Covers a couple of approaches to training an early ranker with knowledge distillation from final ranker.

Files:

  1. baseline_early_ranker.py : Shows how normally early ranker is trained.
  2. kd_aux_early_ranker.py : Shows how we can train using auxiliary tasks corresponding to teacher labels.
  3. kd_shared_early_ranker.py : Uses the shared logits approach to KD.