/cloud-map-reduce-and-distributed-jobs

Distributed Image Processing on Google Cloud Platform using Cloud Dataproc.

Primary LanguageScala

cloud-map-reduce-and-distributed-jobs

Distributed Image Processing on Google Cloud Platform using Cloud Dataproc.

Description:

Used Apache Spark on Cloud Dataproc to distribute a computationally intensive image processing task onto a cluster of machines.

Steps:

  1. Created a development machine in Compute Engine.
  2. Installed software on that machine.
  3. Created a Google Cloud Storage Bucket and collected images.
  4. Created a Cloud Dataproc cluster.
  5. Submitted jobs to Cloud Dataproc.

Here is the link to demo of this project:

VIDEO DEMO

Reference Lab:

Distributed Image Processing in Cloud Dataproc-Qwiklabs
Google Cloud Dataproc