Two types of Densely Connected Convolutional Networks (DenseNets) are available:
- DenseNet - without bottleneck layers
- DenseNet-BC - with bottleneck layers
Each model can be tested on such datasets:
- Cifar10
- Cifar10+ (with data augmentation)
- Cifar100
- Cifar100+ (with data augmentation)
- SVHN
A number of layers, blocks, growth rate, image normalization and other training params may be changed trough shell or inside the source code.
Example run:
python run_dense_net.py --train --test --dataset=C10
List all available options:
python run_dense_net.py --help
There are also many other implementations - they may be useful also.
Citation:
@article{Huang2016Densely,
author = {Huang, Gao and Liu, Zhuang and Weinberger, Kilian Q.},
title = {Densely Connected Convolutional Networks},
journal = {arXiv preprint arXiv:1608.06993},
year = {2016}
}
Test results on various datasets. Image normalization per channels was used. Results reported in paper provided in parenthesis. For Cifar+ datasets image normalization was performed before augmentation. This may cause a little bit lower results than reported in paper.
| Model type | Depth | C10 | C10+ | C100 | C100+ |
|---|---|---|---|---|---|
| DenseNet(k = 12) | 40 | 6.67(7.00) | 5.44(5.24) | 27.44(27.55) | 25.62(24.42) |
| DenseNet-BC(k = 12) | 100 | 5.54(5.92) | 4.87(4.51) | 24.88(24.15) | 22.85(22.27) |
Approximate training time for models on GeForce GTX TITAN X GM200 (12 GB memory):
- DenseNet(k = 12, d = 40) - 17 hrs
- DenseNet-BC(k = 12, d = 100) - 1 day 18 hrs
Difference compared to the original implementation
The existing model should use identical hyperparameters to the original code. If you note some errors - please open an issue.
Also it may be useful to check my blog post Notes on the implementation DenseNet in tensorflow.
- Model was tested with Python 3.4.3+ and Python 3.5.2 with and without CUDA.
- Model should work as expected with TensorFlow >= 0.10. Tensorflow 1.0 support was recently included.
Repo supported with requirements files - so the easiest way to install all just run:
- in case of CPU usage
pip install -r requirements/cpu.txt. - in case of GPU usage
pip install -r requirements/gpu.txt.