sithu31296/Knowledge-Distillation-Pipeline

PyTorch Knowledge Distillation Framework

FastKD

• Introduction
• Features
• Methods Comparison
• Configuration
• Training
• Evaluation
• Inference

Introduction

PyTorch Knowledge Distillation Framework.

Features

Datasets:

• ImageNet

Sample Model:

• Teacher: ResNet50 (from torchvision)
• Student: ResNet18 (from torchvision)

KD Methods:

• Vanilla KD
• TAKD (Coming Soon)
• CRD (Coming Soon)

Features coming soon:

• Native DDP
• Native AMP

Methods Comparison

Coming Soon...

Configuration

Create a configuration file in configs. Sample configuration for ImageNet dataset can be found here. Then edit the fields you think if it is needed. This configuration file is needed for both training and evaluation scripts.

Training

$ python train.py --cfg configs/CONFIG_FILE_NAME.yaml

Evaluation

$ python val.py --cfg configs/CONFIG_FILE_NAME.yaml