innovative-project-jackdaw

1. Project goals

The goal was to create PoC for platform performing streaming calculation using Kafka and Docker in microservices architecture.

This platform allows to perform Big Data Calculation in Streaming mode.

For the final product a web application responsible for data visualization has been created.

We used a dataset representing flights from and to Brasil as an example.

This project consists of three main modules:

Producer
- reads data from file
- sends messages serialized by KafkaAvroSerializer to the FlightData topic on Kafka
Consumer
- reads data from FlightData topic
- uses KafkaAvroDeserializer
- converts data into a geoJSON
- sends it to Visualization topic on Kafka
Visualization
- consumer
  - reads data from Visualization topic
  - sends data to browser through socket
- flight visualization
  - reads data from socket
  - provides live visualization on map

Every module has its defined environment – a separate docker image defined in Dockerfile.

Docker-compose allows to build and run all the services together.

This solution provides containers with all required dependencies installed.

First you need to have Docker installed on your system. Then run docker or create and start docker machine.

Open a command prompt and use the following commands to clone repository and navigate to project directory:

$ git clone https://github.com/nokia-wroclaw/innovative-project-jackdaw.git
$ cd innovative-project-jackdaw

Run docker-compose

$ docker-compose up

$ ./run.sh

$ docker-compose down

or in order to clear Jackdaw images:

$ ./clear.sh

If everything went right, you should be able to access:

Students

Nokia Supervisors