spark-cluster
There are 26 repositories under spark-cluster topic.
minhky2185/healthcare_data_pipeline
An end-to-end data pipeline for building Data Lake and supporting report using Apache Spark.
mgarralda/hadoop-spark-cluster
Repository containing Docker images for create a cluster Spark on Hadoop Yarn.
AIxHunter/Spark-k8s-pod-template
Steps to deploy a Spark app to Kubernetes cluster using spark-submit or a pod template
aimanamri/raspberry-pi4-hadoop-spark-cluster
This is a self-documentation of learning distributed data storage, parallel processing, and Linux OS using Apache Hadoop, Apache Spark and Raspbian OS. In this project, 3-node cluster will be setup using Raspberry Pi 4, install HDFS and run Spark processing jobs via YARN.
SinghHarshita/Clustering-Algorithms-Spark
KMeans, Cure and Canpoy algorithms are demonstrated using Pyspark.
shuaicj/spark-cluster
A spark cluster based on docker-compose.
vaibhavmagon/Spark-Python-MovieReviews
Script to run and find similarities between movies from a movie lens data set using Python & Spark Clustering.
ayseirmak/DistributedFraudDetection
In this study, we propose to use a distributed storage and computation system in order to track money transfers instantly. In particular, we keep our transaction history in a distributed file system as a graph data structure. We try to detect illegal activities by using Graph Neural Networks (GNN) in distributed manner.
karamolegkos/Diastema
This is my contribution in the project Diastema
longNguyen010203/Spark-Processing-AWS
👷🌇 Set up and build a big data processing pipeline with Apache Spark, 📦 AWS services (S3, EMR, EC2, IAM, VPC, Redshift) Terraform to setup the infrastructure and Integration Airflow to automate workflows🥊
dazayzeh/Installing-spark-standalone-to-a-cluster-manually
I'll walk you through launching a cluster manually using Spark standalone deploy mode, as well as connecting an app to the cluster, launching the app, where to view the monitoring and logging.
matthieuvion/spark-cluster
Steps to deploy a local spark cluster w/ Docker. Bonus: a ready-to-use notebook for model prediction on Pyspark using spark.ml Pipeline() on a well known dataset
shuaicj/spark-cluster-zk
A spark cluster containing multiple spark masters based on docker-compose.
aiden-dai/ai-cluster
Start clusters in virtualbox VMs
DanMolenhouse/Distributed-Systems-Project5-Hadoop-and-Spark
In this project, we used both Hadoop / MapReduce and Spark to do distributed computing. The first task was to perform a series of operations using a Mapper and Reduce java file that was implemented on a Hadoop server. The second task was to perform similar operations, but on Spark instead.
harshkavdikar1/GeoSpatial-DataAnalysis-With-Spark
A distributed application to identify top 50 taxi pickup locations in New York by analyzing over 1 billion records using apache spark, hadoop file system and scala.
pientaa/opening-black-box
Deep dive into Spark UDFs' characteristics.
RammySekham/spark-kb
Spark standalone architecture, local architecture and reading hadoop file formats i.e. avro, parquet and ORC
silencebingo/hadoop-spark-cluster
A Hadoop and Spark Cluster on Docker
Turnipdo/Spark-Standalone-Cluster-Setup
To facilitate the initial setup of Apache Spark, this repository provides a beginner-friendly, step-by-step guide on setting up a master node and two worker nodes.
ansjin/docker-spark
docker spark standalone
euiyounghwang/spark_job_interface_service
spark_job_interface_service
flaviostutz/spark-submit-scala
Spark submit extension from bde2020/spark-submit for Scala with SBT
kumarvna/terraform-azurerm-hdinsight
Terraform module to create managed, full-spectrum, open-source analytics service Azure HDInsight. This module creates Apache Hadoop, Apache Spark, Apache HBase, Interactive Query (Apache Hive LLAP) and Apache Kafka clusters.
minsusun/deploy-spark-cluster
configs for deploying the spark clusters on docker and k8s !!