/spark-continuous-streaming

Spark 2.3. End to End Avro Continous Structured Streaming Kafka demo using Twitter´s bijection in Java.

Primary LanguageJavaBSD 3-Clause "New" or "Revised" LicenseBSD-3-Clause

Continous Structured Streaming Avro demo

Spark 2.3. End to End Avro Continous Structured Streaming Kafka demo using Twitter´s bijection in Java.

Requirements

  • Kafka: 1.0.0
  • Java 8
  • Maven

Architecture

Architecture

Usage

Configure kafka hosts and execute both the ProducerConsumerDemo and the StructuredDemo classes.

Latency measures can be seen on ProducerConsumerDemo's output.