This repository is meant to include my experiences with mpi4py.
All nodes in the computing cluster have to running the same version of python3, mpi4py, openmpi, etc.
To keep things simple, the following steps were done on Ubuntu 20.04 LTS and its respective Ubuntu Server.
sudo apt update && sudo apt -y full-upgrade
sudo apt install python3-mpi4py
adduser mpiuser- The master node has to have a SSH key to all slave nodes. To generate a SSH key,
ssh-keygen -t rsa
# Repeat the following for every slave node
cat ~/.ssh/id_rsa.pub | ssh mpiuser@IP-ADDRESS "mkdir ~/.ssh; cat >> .ssh/authorized_keys"- The master node requires a list of all IP addresses of the nodes (including itself).
nano machines-
A copy of the script to be run must be on each node with the exact same path and name. The
copyscript in this repository is meant to make this process much easier, using themachinesfile made in step 2. -
To run a script:
# To run as mpiuser, prefix `sudo -u mpiuser` to the command
mpiexec -hostfile machines --use-hwthread-cpus python3 [absolute path of script]