/Parallel-Deep-Learning-in-Python

Parallelizing Deep Learning using MPI and GPU.

Primary LanguagePython

Parallelizing Deep Learning using MPI and GPU

Implemented a sequential and parallel neural network model using data based parallelism in Python using MPI and GPU computing. Measured the improvement in performance and speed up in training time.

  • mnist-nn.py: 3-layer neural network as digit recognizer (MNIST).
  • mnist-nn-gpu.py: mnist-nn with GPU computing.
  • mnist-nn-data-parallelism.py: mnist-nn with data parallelism (MPI and GPU).