/iot-traffic-monitor

Sample IoT application simulating a monitoring dashboard for a fleet management/shipping company.

Primary LanguageJavaApache License 2.0Apache-2.0

IoT Traffic Monitor

Below is the architecture diagram for IoT Traffic Monitor application. Read the article at InfoQ

IoT Traffic Monitor Architecture

Traffic Monitor application uses following tools and technologies.

  • JDK - 1.8
  • Maven - 3.3.9
  • ZooKeeper - 3.4.13
  • Kafka - 2.1.1
  • Cassandra - 3.0.0
  • Spark - 1.6.2 Pre-built for Hadoop 2.6
  • Spring Boot - 1.3.5
  • jQuery.js
  • Bootstrap.js
  • Sockjs.js
  • Stomp.js
  • Chart.js

IoT Traffic Monitor is a Maven Aggregator project. It includes following three projects.

  • IoT Kafka Producer
  • IoT Spark Processor
  • IoT Spring Boot Dashboard

For building these projects it requires following tools. Please refer README.md files of individual projects for more details.

  • JDK - 1.8
  • Maven - 3.3.9

Setup Application

  1. git https://github.com/jatin7/iot-traffic-monitor.git
  2. Build the required binaries.
cd iot-traffic-monitor
mvn package

  1. Create DataStax/Cassandra Tables

    Create the keyspaces and tables by running the following command.

cqlsh datastax-iot 9042 -f resources/IoTData.cql
  1. Do the following to run Kafka and related components:
Download confluent-5.2.1-2.12.tar.gz from Confluent Website 
mv cconfluent-5.2.1-2.12.tar.gz?dl=0 confluent-5.2.1-2.12.tar.gz
tar xvzf confluent-5.2.1-2.12.tar.gz

vi ~/.bashrc 
export PATH=$PATH:~/apps/confluent-5.2.1/bin

source ~/.bashrc

confluent start ksql-server
confluent status

The output for the confluent status should look like

control-center is [DOWN]
ksql-server is [UP]
connect is [DOWN]
kafka-rest is [DOWN]
schema-registry is [UP]
kafka is [UP]
zookeeper is [UP]

Note: It is required that the DOWN components in this list are not actually enabled.

  1. Create the origin Kafka topic
kafka-topics --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic iot-data-event

Running the application

  1. Start the data producer.
cd iot-kafka-producer
nohup java -jar target/iot-kafka-producer-1.0.0.jar &
  1. Start Spark the data processing application
cd  iot-spark-processor
dse spark-submit --packages org.apache.spark:spark-streaming-kafka_2.11:1.6.3 --class "com.iot.app.spark.processor.IoTDataProcessor" target/iot-spark-processor-1.0.0.jar
#nohup java -jar target/iot-spark-processor-1.0.0.jar &
  1. Start the UI application.
cd iot-springboot-dashboard
nohup java -jar target/iot-springboot-dashboard-1.0.0.jar &```

4. Now open the dashboard UI in a web browser. The application will refresh itself periodically.
```sh
http://localhost:8080