cmftall/spark-config-example

Configuring a Spark application with Typesafe config (example application)

Configuring a Spark application with Typesafe config (example application)

Example code for my blog post "Configuring Spark application with Typesafe config".

Basic instructions

Build assembly JAR:

sbt assembly

Run (local mode):

spark-submit --master local[*] --class xyz.florentforest.sparkconfigexample.ConfigWordCount --files wordcount.conf --driver-java-options -Dconfig.file=wordcount.conf sparkconfigexample-assembly-1.0.jar