EdwardNgo/Tiki-Data-Analysis-V2

Pipeline for getting the tiki data to analysis some attributes and making some dashboards

Tiki-Data-Analysis-V2

Pipeline for getting the tiki data to analysis some attributes and making some dashboards

Contents

• Quick Start
• Tech
• Reference

Quick Start

Cluster overview

Application	URL
Hadoop	localhost:9870
MapReduce	localhost:8089
Elasticsearch Cluster	localhost:27017
Kafka Cluster	localhost:9000
JupyterLab	localhost:8888
Spark Master	localhost:8080

Prerequisites

• Install Docker and Docker Compose

Build from Docker Hub

1. Download the source code or clone the repository
2. Build the cluster

./init.sh
docker-compose up -d

3. Remove the cluster by typing

docker-compose down

Tech

The project was deployed with only my laptop but simulate the cluster with Docker

Hadoop

Apache Spark Standalone Cluster

Elasticsearch Cluster

Kafka Cluster

References

• André Perez
• Big Data Europe - Hadoop Docker
• Big Data Europe - Spark Docker
• Elasticsearch Cluster with Docker
• Setup Kafka Cluster with Docker