PCGrad
This repository contains code for Gradient Surgery for Multi-Task Learning in TensorFlow v1.0+ (PyTorch implementation forthcoming).
PCGrad is a form of gradient surgery that projects a task’s gradient onto the normal plane of the gradient of any other task that has a conflicting gradient, which achieves substantial gains in efficiency and performance on a range of supervised multi-task learning and multi-task reinforcement learning domains. Moreover, it is model-agnostic and can be combined with previously-proposed multitask architectures for enhanced performance.
Installation
Clone this repo and copy PCGrad_tf.py to your codebase.
Usage
optimizer = PCGrad(tf.train.AdamOptimizer()) # wrap your favorite optimizer
losses = # a list of per-task losses
assert len(losses) == num_tasks
train_op = optimizer.minimize(losses)Experiments
Our experiments in the paper were based on the following repositories.
CIFAR-100-MTL: RoutingNetworks
NYUv2: MTAN
MultiMNIST: MultiObjectiveOptimization
MT10/MT50/goal-conditioned pushing in MetaWorld: softlearning with modifications (per-task temperature and per-task replay buffers). We will release modified multi-task softlearning code soon.
Reference
Please cite as:
@article{yu2020gradient,
title={Gradient surgery for multi-task learning},
author={Yu, Tianhe and Kumar, Saurabh and Gupta, Abhishek and Levine, Sergey and Hausman, Karol and Finn, Chelsea},
journal={arXiv preprint arXiv:2001.06782},
year={2020}
}