Hydra — a Multi-Task Learning Framework
- Naive — a separate optimizer for each task
- Gradients averaging — average out the gradients to the network's body
- MGDA — described in the paper Multi-Task Learning as Multi-Objective Optimization (NIPS 2018)
A comprehensive survey on these algorithms (and more) can be found in this blog article.
The code was written on
Python 3.6. Clone this repository:
git clone https://github.com/hav4ik/Hydra
It is recommended to use anaconda for installation of core packages (since
condapackages comes with low-level libraries that can optimize the runtime):
conda install pytorch torchvision cudatoolkit=10.0 -c pytorch conda install numpy pandas scikit-learn
Some of the packages are not available from anaconda, so you can install them using
pip install -r requirements.txt
./starter.sh naive 50
- Proper framework documentation and examples.