iskyzh/hadoop-spark-job

Course project for CS236

hadoop-spark-job

Inside this repo, we:

• one-click setup a hadoop / spark cluster hadoop-docker
  • with the help of https://github.com/big-data-europe/docker-hadoop
  • and https://github.com/big-data-europe/docker-hadoop
• use map-reduce to sort and group temperature data map-reduce-temperature
• use spark to to sort and group temperature data spark-temperature