DisCo: Remedy Self-supervised Learning on Lightweight Models with Distilled Contrastive Learning(ECCV-2022 Oral)
This repository contains the Official Pytorch Implementation for DisCo: Remedy Self-supervised Learning on Lightweight Models with Distilled Contrastive Learning
@article{gao2021disco,
title={DisCo: Remedy Self-supervised Learning on Lightweight Models with Distilled Contrastive Learning},
author={Yuting Gao, Jia-Xin Zhuang, Shaohui Lin, Hao Cheng, Xing Sun, Ke Li, Chunhua Shen},
journal={European Conference on Computer Vision(ECCV)},
year={2022}
}
If the project is useful to you, please give us a star. ⭐️
| Architecture | Self-supervised Methods | Model Checkpoints |
|---|---|---|
| ResNet152 | MoCo-V2 | Model |
| ResNet101 | MoCo-V2 | Model |
| ResNet50 | MoCo-V2 | Model |
For teacher models such as ResNet-50*2 etc, we use their official implementation, which can be downloaded from their github pages.
| Teacher/Students | Efficient-B0 | ResNet-18 | Vit-Tiny | XCiT-Tiny |
|---|---|---|---|---|
| ResNet-50 | Model | Model | - | - |
| ResNet-101 | Model | Model | - | - |
| ResNet-152 | Model | Model | - | - |
| ResNet-50*2 | Model | Model | - | - |
| ViT-Small | - | - | Model | - |
| XCiT-Small | - | - | - | Model |
-
Python3
-
Pytorch 1.6+
-
Detectron2
-
8 GPUs are preferred
-
ImageNet, Cifar10/100, VOC, COCO
Commands can be found on Reproduction.
Code heavily depends on MoCo-V2, Detectron2.