OndrejKucera/knowledge_spark

Notes about Apache Spark (early stage)

Knowledge - Spark

Note: The notes are not exhausted knowledge about Apache Spark.

1. Introduction
2. Architecture
   • Partitioning
   • Spark Execution
   • Scheduling
   • Shuffling
   • Optimizer
3. RDD
   • Action
   • Transformation
   • Persisting data
   • PairRDD
4. Spark SQL - Structured API
   • Schema
   • DataFrames
   • Dataset
   • Deferences between APIs
   • UDFs
5. Join
6. Key/Value data
7. Testing
8. Spark Streaming

... still in progress!

The main source for these notes is Spark: The Definitive Guide and High Performace Spark and Coursera course Big Data Analysis with Scala and Spark