This is a boilerplate project of Spark 3.3.1 with Scala 2.13.10 and sbt. The goal of this repository is to reuse the structure and configuration for future projects.
To test the configuration, this sample project reads a CSV (mock data) using a Spark DataFrame and show some content.
- Spark 3.3.1
- Java 11.
- Spark runs on Java 8/11/17 and Scala 2.12/2.13. This sample use Java 11 because there is a bug in Java 17 with Spark 3.3.1.
- Scala 2.13.10
- sbt 1.8.2
sbt run compile