/Map-Reduce-Framework-Using-Python

Primary LanguagePython

Yet Another Map Reduce (YaMR)

About

The purpose of this project is to implement the main core components of Hadoop's Map Reduce Framework. Three types of operations were worked on during this project, they are :-

  1. WRITE : Given an input file, split it into multiple partitions and store it across multiple worker nodes.
  2. READ : Given a file name, read the different partitions from different workers and display it to the user.
  3. MAP-REDUCE : Given an input file, a mapper file and a reducer file, execute a MapReduce Job on the cluster.

Procedure to run the code :-

  1. Git Clone the Url - git clone https://github.com/Projects-RR-2022/BD1_814_824_830_831
  2. Install File Split - pip install file split
  3. To run the Framework, go to the master file and run the start-all.py file
  4. To run the Operations, go to the client file and run the client_v1.py file

By

  • Neeraj Gopalakrishnan