This repo demonstrates our WirelessFL1, an Internet of Things platform for Federated Learning. We constructed this platform to validate the performance of our proposed FL algorithm in real-world environment.
The preferred approach for installing all the dependencies is to use Anaconda, which is a Python distribution that includes many of the most popular Python packages for science, math, engineering and data analysis. Once you install Anaconda you can run the following command inside the working directory to install the required packages for FL server:
conda env create -f wirelessFL.yml
Once you have all the packages installed, run the following command to activate the environment when you work on the server.
conda activate wirelessFL
Our proposed algorithm will call MATLAB's fmincon()
function to help solve optimization problem. Thus, you need to install necessary MATLAB package according to Install MATLAB Engine API for Python.
We used EMNIST
dataset for prototype environment. To download data and generate non-i.i.d. dataset, run the following command from the data/EMNIST
directory:
python generate_random_niid.py
The program will generate 40-client niid dataset by default. You can modify NUM_USER
vairabke in generate_random_niid.py
to change the number of users.
You could create necessary helper folders by running the following command:
./setup.sh
The wireless communication is achieved in WiFi environment. A powerful WiFi router is required to support 40 or more clients' data transition. Before running experiment, you need to:
- Turn on the WiFi router;
- Connect the server (a laptop) to that WiFi;
- Connect all 40 Raspberry Pis to that WiFi.
[OPTIONAL] To save time, we provide two tips to quickly configure WiFi for Raspberry Pi :
- Automatically connect the Raspberry Pi to WiFi2
- Bind Raspberry Pi to fixed IP address by following steps:
- Open WiFi configuration page in browser (URL:
admin:admin@[IP OF ROUTER]
) - In
PORT Management / DHCP Setting
- Scan devices under this WiFi
- Check device's connect by identity
- In
VLAN: address binding
- Bind address of server and Pis to a fixed ip
- Open WiFi configuration page in browser (URL:
The end-to-end communication is implemented based on TCP protocol. We used socket programing in Python to achieve low-level speed monitoring. To ensure clients can connect to the server, you need to change IP
variable in src/communication/comm_config.py
to the IP address of server. The following shows the default configuration:
# Connection
IP="192.168.43.2"
Since the procedure of configuring IoT devices is tedious, we prepared two bash scripts to help automatically set up the clients and execute programs on Raspberry Pi remotely.
To run programs on Raspberry Pi, you need to install necessary packages and send the Python code to them. We prepared a prepare.sh
bash script to automatically do that for all 40 clients. First, you need to write IP addresses of clients in the script like following:
HOSTS="192.168.XX.01 192.168.XX.02"
Then, you can run the following command to install package and send the latest version of code to clients:
sh prepare.sh
We also prepared a script for you to remotely run client programs on Raspberry Pi for federated learning training. Like previous step, you need to first add IP address in the run.sh
script:
HOSTS="192.168.XX.01 192.168.XX.02"
First, run server by
python app.py --model server
Then, run clients by
./run.sh
or [optional] python app.py --model client
To estimate
args['algo'] = 'weighted'
fedServer.start_experiment()
fl_experiment(args, fedServer)
args['algo'] = 'uniform'
fedServer.start_experiment()
fl_experiment(args, fedServer)
Then, esitmate by execute cells in estimate.ipynb
After that, you can run experiments on proposed and statistical sampling to compare efficiency of shcemes:
args['algo'] = 'proposed'
fedServer.start_experiment()
fl_experiment(args, fedServer)
args['algo'] = 'statistical'
fedServer.start_experiment()
fl_experiment(args, fedServer)
Footnotes
-
The platform is constructed is based on the works in fedavgpy and FedProx. ↩
-
The following two links are helpful for auto connect Pis to WiFi: 1. How To: Connect your Raspberry Pi to WiFi; 2. Automatically connect a Raspberry Pi to a Wifi network. ↩