/spring-kafka-de-serialization-types

The goal is to play with Spring Kafka. We've implemented 5 examples of producer and consumer services that exchanges messages through Kafka using different types of serialization and approaches.

Primary LanguageJava

spring-kafka-de-serialization-types

The goal of this project is to play with Spring for Apache Kafka. We've implemented 5 examples of producer and consumer services that exchanges messages through Kafka using different types of serialization and approaches.

Proof-of-Concepts & Articles

On ivangfr.github.io, I have compiled my Proof-of-Concepts (PoCs) and articles. You can easily search for the technology you are interested in by using the filter. Who knows, perhaps I have already implemented a PoC or written an article about what you are looking for.

Examples

The following examples demonstrate a producer that pushes "News" messages to a topic in Kafka and a consumer that listens those messages from Kafka.

Example Diagram
string-de-serialization project-diagram-samples
json-de-serialization project-diagram-samples
avro-de-serialization project-diagram-samples
avro-2-de-serialization project-diagram-samples
avro-3-de-serialization project-diagram-samples

Prerequisites

Start Environment

  • Open a terminal and inside spring-kafka-de-serialization-types root folder run:

    docker compose up -d

  • Wait for Docker containers to be up and running. To check it, run:

    docker compose ps

Useful Links

  • Schema Registry UI

    Schema Registry UI can be accessed at http://localhost:8001

  • Kafka Topics UI

    Kafka Topics UI can be accessed at http://localhost:8085

  • Kafka Manager

    Kafka Manager can be accessed at http://localhost:9000

    Configuration

    • First, you must create a new cluster. Click on Cluster (dropdown on the header) and then on Add Cluster;
    • Type the name of your cluster in Cluster Name field, for example: MyCluster;
    • Type zookeeper:2181 in Cluster Zookeeper Hosts field;
    • Enable checkbox Poll consumer information (Not recommended for large # of consumers if ZK is used for offsets tracking on older Kafka versions);
    • Click on Save button at the bottom of the page.

Shutdown

To stop and remove docker compose containers, network and volumes, go to a terminal and, inside spring-kafka-de-serialization-types root folder, run the command below:

docker compose down -v

Cleanup

To remove the Docker created by this project, go to a terminal and, inside spring-kafka-de-serialization-types root folder, run the following script:

./remove-docker-images.sh all

References