/spark-streaming-twitter

Sample Spark Streaming

Primary LanguagePython

spark-streaming-sample

Sample Spark Streaming

Setup

  1. Install Spark (spark 2.3.0 with hadoop 2.7 or later)
  2. sudo tar -xvzf spark-2.3.0-bin-hadoop2.7.tgz
  3. Update the ~/.bash-profile with environment varibles
  4. run source ~/.bash-profile
  5. run pyspark to check if the installation is working proper or not

Running each file

  1. On one terminal run nc -l 9999
  2. On another terminal run spark-submit "filename".py localhost 9999