xhchrn/LG-FedAvg

Federated Learning with Local and Global Representations, NeurIPS 2019 FL workshop

Federated Learning with Local and Global Representations

Pytorch implementation for federated learning with local and global representations.

Correspondence to:

• Paul Liang (pliang@cs.cmu.edu)
• Terrance Liu (terrancl@cs.cmu.edu)

Paper

Think Locally, Act Globally: Federated Learning with Local and Global Representations
Paul Pu Liang*, Terrance Liu*, Liu Ziyin, Ruslan Salakhutdinov, and Louis-Philippe Morency
NeurIPS 2019 Workshop on Federated Learning (distinguished student paper award). (*equal contribution)

If you find this repository useful, please cite our paper:

@article{liang2020think,
  title={Think locally, act globally: Federated learning with local and global representations},
  author={Liang, Paul Pu and Liu, Terrance and Ziyin, Liu and Salakhutdinov, Ruslan and Morency, Louis-Philippe},
  journal={arXiv preprint arXiv:2001.01523},
  year={2020}
}

Installation

First check that the requirements are satisfied:
Python 3.6
torch 1.2.0
torchvision 0.4.0
numpy 1.18.1
sklearn 0.20.0
matplotlib 3.1.2
Pillow 4.1.1

The next step is to clone the repository:

git clone https://github.com/pliang279/LG-FedAvg.git

Data

We run FedAvg and LG-FedAvg experiments on MNIST (link) and CIFAR10 (link). See our paper for a description how we process and partition the data for federated learning experiments.

FedAvg

Results can be reproduced running the following:

MNIST

python main_fed.py --dataset mnist --model mlp --num_classes 10 --epochs 1000 --lr 0.05 --num_users 100 --shard_per_user 2 --frac 0.1 --local_ep 1 --local_bs 10 --results_save run1

CIFAR10

python main_fed.py --dataset cifar10 --model cnn --num_classes 10 --epochs 2000 --lr 0.1 --num_users 100 --shard_per_user 2 --frac 0.1 --local_ep 1 --local_bs 50 --results_save run1

LG-FedAvg

Results can be reproduced by first running the above commands for FedAvg and then running the following:

MNIST

python main_lg.py --dataset mnist --model mlp --num_classes 10 --epochs 200 --lr 0.05 --num_users 100 --shard_per_user 2 --frac 0.1 --local_ep 1 --local_bs 10 --num_layers_keep 3 --results_save run1 --load_fed best_400.pt

CIFAR10

python main_lg.py --dataset cifar10 --model cnn --num_classes 10 --epochs 200 --lr 0.1 --num_users 100 --shard_per_user 2 --frac 0.1 --local_ep 1 --local_bs 50 --num_layers_keep 2 --results_save run1 --load_fed best_1200.pt

MTL

Results can be reproduced running the following:

MNIST

python main_mtl.py --dataset mnist --model mlp --num_classes 10 --epochs 1000 --lr 0.05 --num_users 100 --shard_per_user 2 --frac 0.1 --local_ep 1 --local_bs 10 --num_layers_keep 5 --results_save run1

CIFAR10

python main_mtl.py --dataset cifar10 --model cnn --num_classes 10 --epochs 2000 --lr 0.1 --num_users 100 --shard_per_user 2 --frac 0.1 --local_ep 1 --local_bs 50 --num_layers_keep 5 --results_save run1

If you use this code, please cite our paper:

@article{liang2019_federated,
  title={Think Locally, Act Globally: Federated Learning with Local and Global Representations},
  author={Paul Pu Liang and Terrance Liu and Ziyin Liu and Ruslan Salakhutdinov and Louis-Philippe Morency},
  journal={ArXiv},
  year={2019},
  volume={abs/2001.01523}
}

Acknowledgements

This codebase was adapted from https://github.com/shaoxiongji/federated-learning.