techmonad/learning-spark

This is sample project for learning spark by examples. This project have many working examples of RDD, DataFrame, DataSet.

learning-spark

Run dataset operations

 sbt "runMain com.techmonad.learn.dataset.DataSetOps"

Run dataframe operations

 sbt "runMain com.techmonad.learn.DataFrameOps"

Run RDD operations

 sbt "runMain com.techmonad.learn.RDDOps"

Run Structured Streaming operations

// start natcat server and paste some texts
 nc -lk 9999

 //Now start streaming app
 sbt "runMain com.techmonad.learn.StructuredStreamingOps"