mpi4tf
This is a basic distributed training enabled platform for training Tensorflow.
Pre-requisites
Install OpenMPI 4.x.x or any other MPI implementation.
Install via Pip
pip3 install mpi4tf
Install From Source
Clone the mpi4tf
repo then,
python3 setup.py install
Development Mode
In the development mode use the following command to build the libraries.
python3 setup.py develop
Test
mpirun -n 4 python3 test/test_mpi.py
MNIST Data Parallel Demo
Run with Parallelism 4
./bin/run_mnist_dist.sh 4
Notes
As this is a MPI backend you can use all the MPI flags to add different functionality in running experiments.