Image-Backbones-Implementations

My implementations of image backbones with PyTorch.

Training

accelerate-launch main.py [-c CONFIG] [-e EXP_DIR] [--xxx.yyy zzz ...]

This repo uses the 🤗 Accelerate library for multi-GPUs/fp16 supports. Please read the documentation on how to launch the scripts on different platforms.
Results (logs, checkpoints, tensorboard, etc.) of each run will be saved to EXP_DIR. If EXP_DIR is not specified, they will be saved to runs/exp-{current time}/.
To modify some configuration items without creating a new configuration file, you can pass --key value pairs to the script. For example, the default optimizer in ./configs/resnet18_cifar10.yaml is SGD, and if you want to change it to Adam without bothering to create a new file, you can simply pass --train.optim.type Adam.

For example, to train resnet18 on CIFAR-10:

accelerate-launch main.py -c ./configs/resnet18_cifar10.yaml

models	#params	MACs	acc@1(%)
VGG-11	9.20M	0.15G	90.97
VGG-19 (BN)	20.0M	0.40G	94.00
ResNet-18	11.2M	5.59G	95.64
PreActResNet-18	11.2M	5.59G	95.45
ResNeXt-29 (32x4d)	4.78M	6.90G	95.16
SE-ResNet-18	11.3M	5.59G	95.65
CBAM-ResNet-18	11.3M	5.59G	95.49
MobileNet	3.22M	0.48G	92.09
ShuffleNet 1x (g=8)	0.91M	0.50G	92.82
ViT-Tiny/4	5.36M	0.37G	85.66

Note: MACs are calculated by fvcore library.

All the ConvNets are trained with the following settings:

training duration: 64k steps
batch size: 256
learning rate: start with 0.1, end with 0.001 using a cosine annealing scheduler, no warm-up
optimizer: SGD, weight decay 5e-4, momentum 0.9

The ViTs are trained with the following settings:

training duration: 64k steps
batch size: 512
learning rate: start with 0.001, end with 0.00001 using a cosine annealing scheduler, no warm-up
optimizer: Adam, weight decay 5e-5, betas (0.9, 0.999)