Process_Based_Parallelism_and_Message_Passing_Interface

Process Based Parallelism is making programs that run in parallel with an approach using processes (not threads)

Message Passing is used in Distributed Memory. In message passing, the program that runs in one pair (core-memory) is called a process. Two processes can communicate by calling a function, process 1 calls the send function and process 2 calls the receive function. The implementation of message passing is known as Message Passing Interface (MPI). MPI is not a new programming language but MPI is a library that is in several programming languages such as C, Fortran, Python.

In distributed memory, cores can only access their own memory, private memory. From the programmer's point of view, a distributed memory system consists of a collection of pairs memory that is connected to the network and memory associated with the core that can only be accessed directly by that core

need :

install mpiexec (https://www.microsoft.com/enus/download/details.aspx?id=57467)
pip install pip install mpi4py
install python
installm numpy

to run : mpiexec -n 10 py name_of_your_file

kolektif gather

PS A:\GitHub\Pararel> mpiexec -n 10 py .\kolektif_Gather.py
rank : 1data : None
rank : 5data : None
rank : 7data : None
rank : 3data : None
rank : 2data : None
rank : 6data : None
rank : 9data : None
rank : 4data : None
rank : 8data : None
rank : 0data : [1, 4, 9, 16, 25, 36, 49, 64, 81, 100]

Parralel

PS A:\GitHub\Pararel> mpiexec -n 10 py .\parralel_10_.py
rank adalah : 8 dan Size adalah : 10
rank adalah : 4 dan Size adalah : 10
rank adalah : 9 dan Size adalah : 10
rank adalah : 0 dan Size adalah : 10
rank adalah : 1 dan Size adalah : 10
rank adalah : 7 dan Size adalah : 10
rank adalah : 3 dan Size adalah : 10
rank adalah : 6 dan Size adalah : 10
rank adalah : 2 dan Size adalah : 10
rank adalah : 5 dan Size adalah : 10

kolektif broadcast

PS A:\GitHub\Pararel> mpiexec -n 10 py .\kolektif_broadcast.py
proses = 8variable dibagikan = 100
proses = 0variable dibagikan = 100
proses = 1variable dibagikan = 100
proses = 9variable dibagikan = 100
proses = 4variable dibagikan = 100
proses = 5variable dibagikan = 100
proses = 6variable dibagikan = 100
proses = 2variable dibagikan = 100
proses = 7variable dibagikan = 100
proses = 3variable dibagikan = 100

kolektif scatter

PS A:\GitHub\Pararel> mpiexec -n 10 py .\kolektif_scatter.py
rank : 1data : 4
rank : 4data : 25
rank : 2data : 9
rank : 5data : 36
rank : 3data : 16
rank : 6data : 49
rank : 8data : 81
rank : 0data : 1
rank : 7data : 64
rank : 9data : 100

kolektif all to all

PS A:\GitHub\Process_Based_Parallelism_and_Message_Passing_Interface> mpiexec -n 10 py .\kolektif_all_to_all.py
Prosess 4 mengirim [ 0 5 10 15 20 25 30 35 40 45] diterima [ 4 8 12 16 20 24 28 32 36 40]
Prosess 9 mengirim [ 0 10 20 30 40 50 60 70 80 90] diterima [ 9 18 27 36 45 54 63 72 81 90]
Prosess 7 mengirim [ 0 8 16 24 32 40 48 56 64 72] diterima [ 7 14 21 28 35 42 49 56 63 70]
Prosess 5 mengirim [ 0 6 12 18 24 30 36 42 48 54] diterima [ 5 10 15 20 25 30 35 40 45 50]
Prosess 8 mengirim [ 0 9 18 27 36 45 54 63 72 81] diterima [ 8 16 24 32 40 48 56 64 72 80]
Prosess 6 mengirim [ 0 7 14 21 28 35 42 49 56 63] diterima [ 6 12 18 24 30 36 42 48 54 60]
Prosess 3 mengirim [ 0 4 8 12 16 20 24 28 32 36] diterima [ 3 6 9 12 15 18 21 24 27 30]
Prosess 1 mengirim [ 0 2 4 6 8 10 12 14 16 18] diterima [ 1 2 3 4 5 6 7 8 9 10]
Prosess 2 mengirim [ 0 3 6 9 12 15 18 21 24 27] diterima [ 2 4 6 8 10 12 14 16 18 20]
Prosess 0 mengirim [0 1 2 3 4 5 6 7 8 9] diterima [0 0 0 0 0 0 0 0 0 0]

MPI reduction

PS A:\GitHub\Process_Based_Parallelism_and_Message_Passing_Interface> mpiexec -n 10 py .\MPI_reduction.py
proses 1 mengirim [0 2 4]
on task 1 after reduc : data = [0 0 0]
proses 9 mengirim [ 0 10 20]
on task 9 after reduc : data = [0 0 0]
proses 5 mengirim [ 0 6 12]
on task 5 after reduc : data = [0 0 0]
proses 7 mengirim [ 0 8 16]
on task 7 after reduc : data = [0 0 0]
proses 3 mengirim [0 4 8]
on task 3 after reduc : data = [0 0 0]
proses 8 mengirim [ 0 9 18]
on task 8 after reduc : data = [0 0 0]
proses 6 mengirim [ 0 7 14]
on task 6 after reduc : data = [0 0 0]
proses 2 mengirim [0 3 6]
on task 2 after reduc : data = [0 0 0]
proses 4 mengirim [ 0 5 10]
on task 4 after reduc : data = [0 0 0]
proses 0 mengirim [0 1 2]
on task 0 after reduc : data = [ 0 55 110]