Oort

This repository contains scripts and instructions for reproducing the experiments in our OSDI '21 paper "Oort: Efficient Federated Learning via Guided Participant Selection".

Overview

Getting Started
Run Experiments and Validate Results
Repo Structure
Acknowledgements
Contact

Getting Started

Our install.sh will install the following automatically:

Anaconda Package Manager
CUDA 10.2

Note: if you prefer different versions of conda and CUDA, please check comments in install.sh for details.

Run the following commands to install Oort.

git clone https://github.com/SymbioticLab/Oort
cd Oort
source install.sh

Run Experiments and Validate Results

The output of the experiment will validate the following major claims in our evaluation (section 7 in paper):

FL Training:

Oort outperforms existing random participant selection by 1.2×-14.1× in time-to-accuracy performance, while achieving 1.3%-9.8% better final model accuracy (§7.2.1) -> Table 1 and Figure 9.
Oort achieves close-to-optimal model efficiency by adaptively striking the trade-off between statistical and system efficiency with different components (§7.2.2) -> Figure 11 and 12.
Oort outperforms its counterpart over a wide range of parameters and different scales of experiments, while being robust to outliers (§7.2.3) -> Figure 13, 14 and 15.

FL Testing:

Oort can serve developer testing criteria on data deviation while reducing costs by bounding the number of participants needed even without individual data characteristics(§7.3.1) —> Figure 16.
With the individual information, Oort improves the testing duration by 4.7× w.r.t. Mixed Integer Linear Programming (MILP) solver, and is able to efficiently enforce developer preferences across millions of clients (§7.3.2) -> Figure 17.

Training

Please go to ./training directory and follow the training README to run training scripts.

Testing

Please go to ./testing directory and follow the testing README to run testing scripts.

Repo Structure

Repo Root
|---- Oort        # Oort code base.
|---- training
    |---- evals     # Submit/terminate training jobs
        |---- configs   # Configuration examples
|---- testing       # Testing scripts

Notes

please consider to cite our paper if you use the code or data in your research project.

@inproceedings{Oort-osdi21,
  title={Efficient Federated Learning via Guided Participant Selection},
  author={Fan Lai and Xiangfeng Zhu and Harsha V. Madhyastha and Mosharaf Chowdhury},
  booktitle={USENIX Symposium on Operating Systems Design and Implementation (OSDI)},
  year={2021}
}

Acknowledgements

Thanks to Qihua Zhou for his Falcon repo.

Contact

Fan Lai (fanlai@umich.edu) and Xiangfeng Zhu (xzhu0027@gmail.com)

zcysky/Oort