L^2 compression

Introduction

Deep neural networks have delivered remarkable performance and have been widely used in various visual tasks. However, their huge size causes significant inconvenience for transmission and storage. This work proposes a unified post-training model size compression method that combines lossy and lossless compression.

Requirements

torch
torchvision
constriction
ninja
matplotlib
timm

pip install -r requirements.txt

Usage

train

change the model checkpoints paths in train/train_final

state_path = {
    "resnet18": "",
    "resnet50": "",
    "mobilenetv2": "",
    "mnasnet": "",
    "regnetx_600m": "",
    "regnetx_3200m": ""
}

change dataset in train/train_final

train_dataset = ImageNetDataset(ROOTDIR + '/train/', 'train.txt', train_transform)
test_dataset = ImageNetDataset(ROOTDIR + 'val/', METADIR + 'val.txt', val_transform)

the calibration dataset we provide is train.txt in the repo

examples are in run.sh

# example ResNet18
python -u -m train.train_final \
    --model_name resnet18 \
    --lambda_r 1e-6 \
    --lambda_kd 1.0 \
    --weight_transform edgescale \
    --bias_transform scale \
    --transform_iter 300 \
    --transform_lr 0.0001 \
    --reconstruct_iter 1000 \
    --reconstruct_lr 5e-06 \
    --resolution 64 \
    --diffkernel cos \
    --log_path ./log \
    --target_CR 10.0  \
    --run_name resnet_test

encode and decode