DBPRO IoT Framework

Development of an IoT framework for stream based analysis of sensor data. DBPRO, TU Berlin, summer term 2018.

(Original project home: https://gitlab.tubit.tu-berlin.de/anton.rudacov/DBPRO-IoT-Framework)

Features

This application is an example of basic Flink-Kafka-InfluxDB workflow.

The program performs following steps:

generates a stream of timestamps and attaches data to it (captured before from a sensor).
sinks stream data to InfluxDB and to a Kafka topic
consumes data from Kafka topic (in another thread) and sinks it to InfluxDB as well
splits infinite stream in windows and calculates "Dynamic Time Warping" distances in order to recognise the input
adds annotations containing recognised character to Grafana via HTTP request

How to use

Just run the main() method in jobs/App.java

In order to actually get any observable output you have to run the necessary instances (Kafka, InfluxDB, Grafana) and setup their addresses in corresponding java classes.

Built With

Kafka - distributed streaming platform
Apache Flink - data processing engine
InfluxDB - time-series data storage
Grafana - time series analytics platform
Maven - Build automation tool and dependency management
GitLab - Git repository manager, wiki, issue tracking and CI/CD
Docker - Testing and CI environment (container platform)

Authors

Anton Rudacov - @antonrud
Janine Schulte - @janineschulte
Sven Hellweg - @hellweg.sven

License

This project is licensed under the MIT License - see the LICENSE file for details

Acknowledgements

Project structure and some implementations are inspired by this awesome demonstration by Jamie Grier.

Dynamic Time Warping algorithm is created by Cheol-Woo Jung (browse source).