This project is an event driven task management application that comprises of PostgreSQL DB and ReactJS, Apache Kafka, Kafka Streams (based on single writer principle) and the major ecosystem components like SchemaRegistry, Kafka Connect.
The UI is based on ReactJS and Bootstrap.
Keycloak with OpenLDAP is used as IdP (Identity Provider) to enable OAuth2 and OIDC driven AAA (AuthN, AuthZ, Accounting).
The Kafka Stream source, processor and sink implementations are based on Spring Cloud Stream Kafka Binder.
To have a solid baseline and understanding on Apache Kafka ecosystem following e-books are should read references (they are DRM free and freely distributed by Confluent):
As this is a PoC project a local domain is used for Traefik edge router. Following entries should be entered for web and identity provider access in /etc/hosts:
127.0.0.1 web.poc.local
127.0.0.1 idp.poc.local
Here is the architecture model of project:
The application utilizes Spring Boot Actuator endpoint kafkastreamstopology to retrieve topology definition for task-processor Kafka Streams app. The Processor Topology button on header menu renders this in image format using kafka-streams-viz:
The project is shipped with of two different deployment modes as fully containerized deployment and hybrid deployment:
- Fully Containerized: All of the components including SpringBoot based microservices are containerized. Can be started and stopped using shell scripts.
- Hybrid Mode: All of the components except SpringBoot based microservices are containerized. Can be used for development and be started and stopped using shell scripts.
Before starting mvn clean install should be invoked for top-level project in order to generate app binaries. See bin.
Here are the components:
ReactJS based Web UI to provide basic portal to manage tasks. React Hook and Context API are also used for state management:
Event Driven messaging layer using Apache Kafka Ecosystem Tooling:
- Zookeeper
- Kafka Brokers
- Schema Registry
- Kafka Connect
- Rest Proxy
- Kafka Monitoring Stack
PostgreSQL RDBMS to persist event data on user and task topics for viewing on UI. PgAdmin is also available for DB administration.
Prometheus and Grafana based monitoring stack is available to view status of Kafka brokers:
There exists a sample Grafana dashboard in JSON format that can be imported into Grafana.
Keycloak Identity Provider for OAuth2 & OIDC based AAA enablement. The IdP is backed by an OpenLDAP instance which is initiated by a designated user database.
The task user list on Add User screen is retrieved via Keycloak REST API that is populated from LDAP. To enable this functionality Keycloak admin should assign view-users role of realm-management client to added realm's default role set:
Other settings:
There exists a documentation on ksql statements used in this project.
The project brings a statistics page comprised of React Plotly charts fed with real time data from ksqlDB queries. The task distribution graph has the ability to set window size and historical framing (as a baseline to re-playing) based on date and time:
This project contains deployment objects to activate the task manager application on a RHOCP cluster. These objects are derived from objects generated by kompose tool using existing docker-compose.yml file. The objects were tested on local OCP deployment using CodeReady Containers technology. The build and deploy scripts can be used to generate custom images and deploy them on cluster.
