Created by Jinming Cao, Yangyan Li, Mingchao Sun, Ying Chen, Dani Lischinski, Daniel Cohen-Or, Baoquan Chen, and Changhe Tu.
DO-Conv is a depthwise over-parameterized convolutional layer, which can be used as a replacement of conventional convolutional layer in CNNs in the training phase to achieve higher accuracies. In the inference phase, DO-Conv can be fused into a conventional convolutional layer, resulting in the computation amount that is exactly the same as that of a conventional convolutional layer.
Please see our preprint on arXiv for more details, where we demonstrated the advantages of DO-Conv on various benchmark datasets/tasks.
We highly welcome issues, rather than emails, for DO-Conv related questions.
Moreover, it would be great if a minimal reproduciable example code is provide in the issue.
We take the model zoo of GluonCV as baselines. The settings in the baselines have been tuned to favor baselines, and they are not touched during the switch to DO-Conv. In other words, DO-Conv is the one and only change over baselines, and no hyper-parameter tuning is conducted to favor DO-Conv. We consider GluonCV highly reproducible, but still, to exclude clutter factors as much as possible, we train the baselines ourselves, and compare DO-Conv versions with them, while reporting the performance provided by GluonCV as reference. The results are summarized in this table where the “DO-Conv” column shows the performance gain over the baselines.
| Network | Depth | Reference | Baseline | DO-Conv |
|---|---|---|---|---|
| Plain | 18 | - | 69.97 | +1.01 |
| ResNet-v1 | 18 | 70.93 | 70.87 | +0.82 |
| 34 | 74.37 | 74.49 | +0.49 | |
| 50 | 77.36 | 77.32 | +0.08 | |
| 101 | 78.34 | 78.16 | +0.46 | |
| 152 | 79.22 | 79.34 | +0.07 | |
| ResNet-v1b | 18 | 70.94 | 71.08 | +0.71 |
| 34 | 74.65 | 74.35 | +0.77 | |
| 50 | 77.67 | 77.56 | +0.44 | |
| 101 | 79.20 | 79.14 | +0.25 | |
| 152 | 79.69 | 79.60 | +0.10 | |
| ResNet-v2 | 18 | 71.00 | 70.80 | +0.64 |
| 34 | 74.40 | 74.76 | +0.22 | |
| 50 | 77.17 | 77.17 | +0.31 | |
| 101 | 78.53 | 78.56 | +0.11 | |
| 152 | 79.21 | 79.24 | +0.14 | |
| ResNext | 50_32x4d | 79.32 | 79.21 | +0.40 |
| MobileNet-v1 | - | 73.28 | 73.30 | +0.03 |
| MobileNet-v2 | - | 72.04 | 71.89 | +0.16 |
| MobileNet-v3 | - | 75.32 | 75.16 | +0.14 |
In thie repo, we provide reference implementation of DO-Conv in Tensorflow (tensorflow-gpu==2.2.0), PyTorch (pytorch==1.4.0, torchvision==0.5.0) and GluonCV (mxnet-cu100==1.5.1.post0, gluoncv==0.6.0), as replacement to tf.keras.layers.Conv2D, torch.nn.Conv2d and mxnet.gluon.nn.Conv2D, respectively. Please see the code for more details.
We highly welcome pull requests for adding support for different versions of Pytorch/Tensorflow/GluonCV.
We show how to use DO-Conv based on the examples provided in the Tutorial of TensorFlow with MNIST dataset.
1 . Run the demo example first to get the accuracy of the baseline.
python sample_tf.py
If there is any wrong at this step, please check whether the tensorflow version meets the requirements.
2 . Replace these lines:
self.conv1 = Conv2D(32, 3, activation='relu')
self.conv2 = Conv2D(8, 3, activation='relu')
with
self.conv1 = DOConv2D(32, 3, activation='relu')
self.conv2 = DOConv2D(8, 3, activation='relu')
to apply DO-Conv without any other changes.
python sample_tf.py
3 . We provide the performance improvement in this demo example as follows. (averaged accuracy (%) of five runs)
| run1 | run2 | run3 | run4 | run5 | avg | + | |
|---|---|---|---|---|---|---|---|
| Baseline | 98.5 | 98.51 | 98.54 | 98.46 | 98.51 | 98.504 | - |
| DO-Conv | 98.71 | 98.62 | 98.67 | 98.75 | 98.66 | 98.682 | 0.178 |
4 . Then you can use DO-Conv in your own network in this way.
We show how to use DO-Conv based on the examples provided in the Tutorial of PyTorch with MNIST dataset.
1 . Run the demo example first to get the accuracy of the baseline.
python sample_pt.py
If there is any wrong at this step, please check whether the pytorch and torchvision versions meets the requirements.
2 . Replace these lines:
model = nn.Sequential(
Conv2d(1, 16, kernel_size=3, stride=2, padding=1),
nn.ReLU(),
Conv2d(16, 16, kernel_size=3, stride=2, padding=1),
nn.ReLU(),
Conv2d(16, 10, kernel_size=3, stride=2, padding=1),
nn.ReLU(),
nn.AdaptiveAvgPool2d(1),
Lambda(lambda x: x.view(x.size(0), -1)),
)
with
model = nn.Sequential(
DOConv2d(1, 16, kernel_size=3, stride=2, padding=1),
nn.ReLU(),
DOConv2d(16, 16, kernel_size=3, stride=2, padding=1),
nn.ReLU(),
DOConv2d(16, 10, kernel_size=3, stride=2, padding=1),
nn.ReLU(),
nn.AdaptiveAvgPool2d(1),
Lambda(lambda x: x.view(x.size(0), -1)),
)
to apply DO-Conv without any other changes.
python sample_pt.py
3 . We provide the performance improvement in this demo example as follows. (averaged accuracy (%) of five runs)
| run1 | run2 | run3 | run4 | run5 | avg | + | |
|---|---|---|---|---|---|---|---|
| Baseline | 94.63 | 95.31 | 95.23 | 95.24 | 95.37 | 95.156 | - |
| DO-Conv | 95.59 | 95.73 | 95.68 | 95.70 | 95.67 | 95.674 | 0.518 |
4 . Then you can use DO-Conv in your own network in this way.
We show how to use DO-Conv based on the examples provided in the Tutorial of GluonCV with MNIST dataset.
1 . Run the demo example first to get the accuracy of the baseline.
python sample_gluoncv.py
If there is any wrong at this step, please check whether the mxnet and gluoncv versions meets the requirements.
2 . Replace these lines:
self.conv1 = Conv2D(20, kernel_size=(5,5))
self.conv2 = Conv2D(50, kernel_size=(5,5))
with
self.conv1 = DOConv2D(1, 20, kernel_size=(5, 5))
self.conv2 = DOConv2D(20, 50, kernel_size=(5, 5))
to apply DO-Conv, note that the 'in_channels' in DOConv2D of GluonCV should be set explicitly.
python sample_gluoncv.py
3 . We provide the performance improvement in this demo example as follows. (averaged accuracy (%) of five runs)
| run1 | run2 | run3 | run4 | run5 | avg | + | |
|---|---|---|---|---|---|---|---|
| Baseline | 98.10 | 98.10 | 98.10 | 98.10 | 98.10 | 98.10 | - |
| DO-Conv | 98.26 | 98.26 | 98.26 | 98.26 | 98.26 | 98.26 | 0.16 |
4 . Then you can use DO-Conv in your own network in this way.