This repository contains the source code for the paper
Shengchao Liu, Yingyu Liang, Anthony Gitter. Loss-Balanced Task Weighting to Reduce Negative Transfer in Multi-Task Learning . AAAI-SA 2019.
The figshare appendix is public here.
This repository implements several multi-task learning algorithms, including the Loss-Balanced Task Weighting (LBTW) approach. LBTW dynamically sets tasks weights while training a multi-task neural network.
- Anaconda2-4.3.1
- PyTorch=0.3
- scikit-learn=0.19
Randomly split PubChem BioAssay (PCBA) [1] into 5 folds. PCBA has 128 binary classification tasks.
- 128 Single-Task Learning (STL).
- 1 Multi-Task Learning (MTL).
- 128 Fine-Tuning.
- 2 GradNorm [2].
- 1 RMTL [3].
- 2 Loss-Balanced Task Weighting (LBTW).
[1] Wang, Yanli, Stephen H. Bryant, Tiejun Cheng, Jiyao Wang, Asta Gindulyte, Benjamin A. Shoemaker, Paul A. Thiessen, Siqian He, and Jian Zhang. "Pubchem bioassay: 2017 update." Nucleic acids research 45, no. D1: D955-D963, 2016.
[2] Chen, Zhao, Vijay Badrinarayanan, Chen-Yu Lee, and Andrew Rabinovich. "Gradnorm: Gradient normalization for adaptive loss balancing in deep multitask networks." arXiv preprint arXiv:1711.02257, 2017.
[3] Liu, Shengchao. "Exploration on Deep Drug Discovery: Representation and Learning." University of Wisconsin-Madison, Master’s Thesis TR1854, 2018.
This project is released under the MIT license.
@article{liu2019loss,
title={Loss-Balanced Task Weighting to Reduce Negative Transfer in Multi-Task Learning},
author={Liu, Shengchao and Liang, Yingyu and Gitter, Anthony},
booktitle={Association for the Advancement of Artificial Intelligence (Student Abstract)},
year={2019}
}