Welcome
This project is small back-up for my talk at Java2Days conference in Sofia, November 2018. The goal of the talk is to:
- give the attendees some fundamental Kafka knowledge.
- show a very minimalistic examples with the Kafka API-s.
- demonstrate some real world examples.
Contents
The project consists of:
- A demo usage of the standard Kafka JAVA Producer and Consumer API-s.
- A demo of super simplified usage of the Kafka Streams API.
- A demo of Sping Kafka.
Prerequisites if you want to use the project:
- A running Kafka installation (details below).
- Knowledge of Kafka theory.
- Reasonable IDE - IntelliJ is certainly recommended!
The classes have decent documentation and casual browsing should be enough to grasp the ideas within.
Kafka installation
If you want to install Kafka and make more deeper experiment maybe you would like a native Kafka installation. If you just want a quick and dirty start and you have docker installed then the docker installation might be the way to go.
Docker
If you have a running docker installation this is the quickest way to get started. The current project contains a docker compose file which will get you started. Navigate to the project root and execute (the docker deamon should be running).
docker-compose -f docker/docker-compose.yml up
If everything goes OK then you should be able to successfully execute the SampleKafkaProducer
.
Then you will be able to consume that with SampleKafkaConsumer
.
Native
Installing Kafka natively without some ad-hoc configurations is easy. Just follow this quick start guide.
Common Kafka commands
This is a cheat sheet with some Kafka console tools commands. Just for reference. Assumes a local Kafka installation with a broker at port 9092 and zookeeper at port 2181.
Create a topic
Creates a topic demo
with replication factor of 2 and 3 partitions.
kafka-topics.sh \
--create \
--zookeeper localhost:2181 \
--replication-factor 2 \
--partitions 3 \
--topic demo
Produce to a topic
Produce to topic demo
.
kafka-console-producer.sh \
--topic demo \
--broker-list localhost:9092
Consume from a topic with double values
Consume from topic temperature-fahrenheit
.
kafka-console-consumer.sh \
--bootstrap-server localhost:9092 \
--topic temperature-fahrenheit \
--value-deserializer org.apache.kafka.common.serialization.DoubleDeserializer
Delete a topic
Delete the topic demo
.
kafka-topics.sh \
--delete \
--zookeeper localhost:2181 \
--topic demo
Describe consumer group
Describes the multiple-kafka-consumer
group (topic, partions, lags, etc)
kafka-consumer-groups.sh \
--bootstrap-server localhost:9092 \
--describe \
--group multiple-kafka-consumer
Contributing
The idea of this talk is to be useful for any attendee.
Reasonable suggestions are welcomed and appreciated via pull requests.
If you have some question or remark (before or after the talk) or notice issues do not hesitate to open an issue.