🧙 My experiments with Spark, understanding it's workings under the hood better!
- Reading Spark's query plans
- Data Skew
- Generating a skewed dataset
- Simulating how a skewed dataset looks like
- Solving data skew using AQE and broadcast joins
- Solving data skew (in joins and aggregations) using salting
- Partitioning for high performance data processing: