Huang9495/Transfer-Learning-Library

Transfer Learning Library for Domain Adaptation, Task Adaptation, and Domain Generalization

Introduction

TLlib is an open-source and well-documented library for Transfer Learning. It is based on pure PyTorch with high performance and friendly API. Our code is pythonic, and the design is consistent with torchvision. You can easily develop new algorithms, or readily apply existing algorithms.

The currently supported algorithms include:

Domain Adaptation for Classification

• Domain Adversarial Neural Network (DANN)
• Deep Adaptation Network (DAN)
• Joint Adaptation Network (JAN)
• Adversarial Discriminative Domain Adaptation (ADDA)
• Conditional Domain Adversarial Network (CDAN)
• Maximum Classifier Discrepancy (MCD)
• Adaptive Feature Norm (AFN)
• Margin Disparity Discrepancy (MDD)
• Batch Spectral Penalization (BSP)
• Minimum Class Confusion (MCC)

Partial Domain Adaptation

• Domain Adversarial Neural Network (DANN)
• Partial Adversarial Domain Adaptation (PADA)
• Importance Weighted Adversarial Nets (IWAN)
• Adaptive Feature Norm (AFN)

Open-set Domain Adaptation

• Open Set Domain Adaptation (OSBP)

Domain Adaptation for Semantic Segmentation

• Cycle-Consistent Adversarial Networks (CycleGAN)
• CyCADA: Cycle-Consistent Adversarial Domain Adaptation
• Adversarial Entropy Minimization (ADVENT)
• Fourier Domain Adaptation (FDA)

Domain Adaptation for Keypoint Detection

• Regressive Domain Adaptation for Unsupervised Keypoint Detection (RegDA, CVPR 2021)

Domain Adaptation for Person Re-identification

• Two at Once: Enhancing Learning and Generalization Capacities via IBN-Net (IBN-Net, ECCV 2018)
• Mutual Mean-Teaching: Pseudo Label Refinery for Unsupervised Domain Adaptation on Person Re-identification (MMT, ICLR 2020)
• Similarity Preserving Generative Adversarial Network (SPGAN, CVPR 2018)

Task Adaptation for Image Classification

• Explicit inductive bias for transfer learning with convolutional networks (L2-SP, ICML 2018)
• Catastrophic Forgetting Meets Negative Transfer: Batch Spectral Shrinkage for Safe Transfer Learning (BSS, NIPS 2019)
• DEep Learning Transfer using Fea- ture Map with Attention for convolutional networks (DELTA, ICLR 2019)
• Co-Tuning for Transfer Learning (Co-Tuning, NIPS 2020)
• Stochastic Normalization (StochNorm, NIPS 2020)
• Learning Without Forgetting (LWF, ECCV 2016)
• Bi-tuning of Pre-trained Representations (Bi-Tuning)

Domain Generalization for Classification

• Two at Once: Enhancing Learning and Generalization Capacities via IBN-Net (IBN-Net, ECCV 2018)
• Domain Generalization with MixStyle (MixStyle, ICLR 2021)
• Learning to Generalize: Meta-Learning for Domain Generalization (MLDG, AAAI 2018)
• Invariant Risk Minimization (IRM)
• Out-of-Distribution Generalization via Risk Extrapolation (VREx, ICML 2021)
• Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalization(GroupDRO)
• Deep CORAL: Correlation Alignment for Deep Domain Adaptation (Deep Coral, ECCV 2016)

Domain Generalization for Person Re-identification

• Two at Once: Enhancing Learning and Generalization Capacities via IBN-Net (IBN-Net, ECCV 2018)
• Domain Generalization with MixStyle (MixStyle, ICLR 2021)

We are planning to add

• DA for Object Detection
• TA for text classification

Installation

To use dalib, talib, dglib, and common in other places, you need to install Transfer-Learn,

python setup.py install

Note that we do not support pip install currently.

For flexible use and modification of Transfer-Learn, please git clone the library.

Documentation

You can find the tutorial and API documentation on the website: Documentation. You can also build the doc by yourself following the instructions in http://tl.thuml.ai/get_started/faq.html.

Also, we have examples in the directory examples. A typical usage is

# Train a DANN on Office-31 Amazon -> Webcam task using ResNet 50.
# Assume you have put the datasets under the path `data/office-31`, 
# or you are glad to download the datasets automatically from the Internet to this path
python dann.py data/office31 -d Office31 -s A -t W -a resnet50  --epochs 20

In the directory examples, you can find all the necessary running scripts to reproduce the benchmarks with specified hyper-parameters.

Contributing

We appreciate all contributions. If you are planning to contribute back bug-fixes, please do so without any further discussion. If you plan to contribute new features, utility functions or extensions, please first open an issue and discuss the feature with us.

Disclaimer on Datasets

This is a utility library that downloads and prepares public datasets. We do not host or distribute these datasets, vouch for their quality or fairness, or claim that you have licenses to use the dataset. It is your responsibility to determine whether you have permission to use the dataset under the dataset's license.

If you're a dataset owner and wish to update any part of it (description, citation, etc.), or do not want your dataset to be included in this library, please get in touch through a GitHub issue. Thanks for your contribution to the ML community!

Contact

If you have any problem with our code or have some suggestions, including the future feature, feel free to contact

• Junguang Jiang (JiangJunguang1123@outlook.com)
• Baixu Chen (cbx_99_hasta@outlook.com)
• Mingsheng Long (longmingsheng@gmail.com)

or describe it in Issues.

For Q&A in Chinese, you can choose to ask questions here before sending an email. 迁移学习算法库答疑专区

Citation

If you use this toolbox or benchmark in your research, please cite this project.

@misc{dalib,
  author = {Junguang Jiang, Baixu Chen, Bo Fu, Mingsheng Long},
  title = {Transfer-Learning-library},
  year = {2020},
  publisher = {GitHub},
  journal = {GitHub repository},
  howpublished = {\url{https://github.com/thuml/Transfer-Learning-Library}},
}

Acknowledgment

We would like to thank School of Software, Tsinghua University and The National Engineering Laboratory for Big Data Software for providing such an excellent ML research platform.