DyRes

Models	CIFAR100	ImageNet
AlexNet	53.18%	------
CC_AlexNet	59.51%	------
DY_AlexNet	59.21%	------
WN_AlexNet	59.63%	------
ResNet18	72.51%	------
CC_ResNet18	74.35%	------
DY_ResNet18	73.44%	------
WN_ResNet18	73.03%	------
MobileNetV2	70.57%	------
CC_MobileNetV2	71.07%	------
DY_MobileNetV2	70.56%	------
WN_MobileNetV2	70.96%	------

Default Training Configurations

Parameter	Value
epochs	90
batch	128
learning rate	0.01
update learning rate every	30 epochs
learning rate update factor	0.1
SGD momentum	0.9
SGD weight decay	5e-4

How To Set Up Python and Pip

https://www.python.org/downloads/

How To Set Up the Environment

To install the necessary Python packages for training

pip3 install -r requirements.txt

How To Run the Training

For simplicity, just run

python3 train.py --network some_defined_network

If you want to play around with the hyper-parameters run python3 train.py -h to see the program's flags or arguments.

--network               Some predefined network architecture

-e, --epoch             Number of epochs for training
-b, --batch             Batch size
-l, --lr                Learning rate for SGD
-m, --momentum          Momentum for SGD
-d, --weight-decay      Weight decay for SGD
-s, --step-size         Update the learning rate every x epochs
-g, --gamma             Learning rate update factor. new_lr = old_lr * gamma

--save                  Whether to save network after training
--dataset               Dataset to be trained with, CIFAR100 or ImageNet
--cuda                  Use GPU to train if the flag is used
--ngpu                  Number of GPUs used for training

Another example to run

python3 train.py --network resnet18 -e 120 -b 512 -l 0.1 -m 0.9 -d 0.0005 -s 80 -g 0.1 --dataset cifar100 --cuda

TODO: