/recipes

The Immerok Apache Flink Cookbook is a collection of examples of Apache Flink applications in the format of "recipes". Each recipe explains how you can solve a specific problem by leveraging one or more of the APIs of Apache Flink. The recipes can be extended or provide a basis for solving your requirements with Apache Flink.

Primary LanguageJavaApache License 2.0Apache-2.0

Recipes

Each recipe is a self-contained module.

Recipe overview

  1. Deserializing JSON from Kafka -> Detailed explanation
  2. Continuously Reading CSV Files -> Detailed explanation
  3. Exactly-once with Apache Kafka® -> Detailed explanation
  4. Writing an Application in Kotlin™ -> Detailed explanation
  5. Joining and Deduplicating Data -> Detailed explanation
  6. Splitting Apache Kafka® events -> Detailed explanation
  7. Capturing Late Data -> Detailed explanation
  8. Creating Dead Letter Queues -> Detailed explanation
  9. Using Session Windows -> Detailed explanation
  10. Serializing Timestamped Data -> Detailed explanation
  11. Reading Apache Kafka Headers -> Detailed explanation

Requirements

  • Maven
  • Java 11

IDE setup

Spotless

These instructions assume you use IntelliJ IDEA.

  • Install the latest version of the google-java-format plugin
    • Note: For the time being the custom version used in Flink development is also fine.
  • Enable the google-java-format as explained here
  • Settings/Preferences -> Editor -> Code style
    • Tick "Enabled Editorconfig support"
    • Scala/Kotlin -> Imports -> Clear the import layout at the bottom
      • This is only relevant for the Scala/Kotlin recipes; you can likely skip this and just call spotless from the command-line
  • Settings/Preferences -> Tools -> Actions on Save
    • Enable "Reformat code" and select Java/Scala/Kotlin file types
    • Enable "Optimize imports"

Note: You can also run mvn spotless:apply on the command-line / IDE to reformat the code.