/Optimal-channel-selection-with-discrete-QCQP

Official PyTorch implementation of "Optimal channel selection with discrete QCQP" (AISTATS'2022)

Primary LanguagePythonMIT LicenseMIT

Optimal channel selection with discrete QCQP

This is the code for reproducing the results of the paper Optimal channel selection with discrete QCQP accepted at AISTATS 2022.

Requirements

  • Python 3.7.9
  • PyTorch 1.4.0
  • TorchVision 0.5.0
  • matplotlib 3.3.2
  • cplex 12.10.0 => You need to install academic version (restricted usage for community version).

Download Pretrained Network

  • Download 'pretrained_networks.zip' from google drive.
wget --no-check-certificate -r 'https://docs.google.com/uc?export=download&id=1ZRL84HmoRG28c7IOSDu5gKNwxGcLIsPU' -O pretrained_networks.zip
  • Unzip to generate './pretrained_networks/'.
unzip pretrained_networks.zip

Training ResNet on Cifar-10

Run Pruning Training ResNet (depth 56,32,20) on Cifar10:

python exp/cifar_exp/main.py --mode (mode) --timelimit (timelimit) --baseseed (baseseed) --target_c (target) --arch (arch)
e.g.) python exp/cifar_exp/main.py --mode flops_2D --timelimit 5000 --baseseed 4 --target_c 0.46 --arch resnet20 --use_cplex True
e.g.2) python exp/cifar_exp/main.py --mode flops_2D --timelimit 5000 --baseseed 4 --target_c 0.46 --arch resnet20 --use_cplex False
e.g.3.) python exp/cifar_exp/main.py --mode flops_4D --timelimit 500 --baseseed 4 --target_c 0.46 --gamma 0.5 --arch resnet20 --use_cplex True
e.g.4.) python exp/cifar_exp/main.py --mode flops_4D --timelimit 500 --baseseed 4 --target_c 0.46 --arch resnet20 --use_cplex False
  • (mode) - 'flops_2D', 'flops_4D', 'mem_2D', 'mem_4D'
  • (timelimit) - timelimit for CPLEX. for resnet20 - 5000, resnet32 - 20000, resnet56 - 80000.
  • (baseseed) - (4,5,6) for resnet20, (1,2,3) for resnet32, resnet56
  • (target_c) - ratio of target constraints and max constraints for conv layers. 0~1
  • (gamma) - gamma for 4D. target_c ~ 1
  • (arch) - 'resnet20', 'resnet32', 'resnet56'
  • (use_cplex) - True or False

Greedy solve for the discrete QCQP

We implemented another fast greedy approach to solve discrete QCQP. Please set '(use_cplex)' to be False.