/Spark_Multi_Cloud_Storage_Utils

Spark Read/Write data from/to Multi Cloud utils (GCP, Azure and AWS)

Primary LanguageHTML

Spark_Multi_Cloud_Storage_Utils

Spark Read data from Multi Cloud Utils(GCP,Azure and AWS)

  1. In this repo Read data from Spark with AWS S3,GCS Storage and Azure Data Lake FS
  2. In this repo Write data into Spark with AWS S3,GCS Storage and Azure Data Lake FS

Pre-Requests

  1. AWS Account
  2. GCP Account
  3. Azure Account

Input Source

  1. AWS S3
  2. Google GCS Storage
  3. Azure Storage (Data lake Gen2)

Output Source

  1. AWS S3
  2. Google GCS Storage
  3. Azure Storage (Data lake Gen2)