Hydra — a Multi-Task Learning Framework

Hydra is a flexible multi-task learning framework written in PyTorch 1.0. The following multi-objective optimization algorithms are implemented:

Naive — a separate optimizer for each task
Gradients averaging — average out the gradients to the network's body
MGDA — described in the paper Multi-Task Learning as Multi-Objective Optimization (NIPS 2018)

A comprehensive survey on these algorithms (and more) can be found in this blog article.

Installation

The code was written on Python 3.6. Clone this repository:
```
git clone https://github.com/hav4ik/Hydra
```
It is recommended to use anaconda for installation of core packages (since conda packages comes with low-level libraries that can optimize the runtime):
```
conda install pytorch torchvision cudatoolkit=10.0 -c pytorch
conda install numpy pandas scikit-learn
```
Some of the packages are not available from anaconda, so you can install them using pip:
```
pip install -r requirements.txt
```

Examples of configuration files can be found here. A minimal example is available in starter.sh. Execute it as follows (will train with configurations in configs/toy_experiments/naive.yaml):
```
./starter.sh naive 50
```