This project was created with a focus on showcasing the utilization of the Debezium tool to implement a Change Data Capture (CDC) process. It effectively tracks data changes and synchronizes information with various destinations such as databases, indexers, private data stores, disks, etc.
Change Data Capture (CDC) is a technique used in databases to identify and capture changes (additions, updates, or deletions) made to the data. Instead of continuously scanning the entire database, CDC allows you to track and capture only the data that has changed since the last check. This helps in efficiently managing and replicating updates, reducing the workload on the database and improving overall performance.
Here, you'll discover setup files, docker-compose configurations, and additional resources intended to serve as a motivational starting point for your real-world projects.
- What technologies will you find here?
- mysql sql database
- debezium CDC platform
- rabbitmq Message broker
- kibaETL (Extract Transform and Load)
-
you must have docker engine installed Here's an article introducing container tools for your local machine. Beyond Docker: Emulation, Orchestration and Virtualization in Apple Silicon
I am Mac user, so I like OrbStack
-
Docker Compose is used as a convenient method to effortlessly bootstrap this application.
- Clone this repo:
git clone [git@github.com:edsonma/demo_cdc_mysql-debezium-rabbitmq-kibaetl.git](git@github.com:edsonma/demo_cdc_mysql-debezium-rabbitmq-kibaetl.git)- Walk to cloned directory:
cd demo_cdc_mysql-debezium-rabbitmq-kibaetl- LGTM (use docker-compose)
docker compose up -dEmbrace collaboration with coding hands. "Contribute your skills, create impact, and collaboratively shape the future of technology!"
Based on Gabriel Bueno CDC Demo
Thank you Man!
Prosit!
.~~~~.
i====i_
|cccc|_)
|cccc|
`-==-'