Event Transformation and Storage Pipeline

This is project is supposed to be a Rube Goldberg Machine. Different aspects of application components are to be added here. The project is never supposed to end. However, it should have "releases" that demonstrate the various capabilities of the application.

What capabilities to expect from here

  • Java
  • Spring
  • Kafka
  • CockroachDB
  • Liquibase
  • Docker and K8s
  • Security features such as Kerberos or LDAP
  • Load testing capabilities
  • Code Coverage via Sonar?
  • Swagger page for APIs
  • AWS
  • Monitoring tools Prometheus and Grafana
  • Terraform
  • Uploading event file and resume uploads when n/w drops(using htmx or angular) Current technologies adapted

The goal is to make deployments and test functionalities with a click of button using dashboards.

Stages of Deployment(Predicted)

  • Local machine(My PC: 32 GB , 6 cores)
  • Deployment as container in Local machine(using Docker Desktop)
  • Deployment in single-node K8s(using Docker Desktop)
  • Deployment in AWS(using Terraform hopefully)

Branch naming convention

We are using GitHub project for creating issue and merging changes.

How often would be the updates

Hopefully regularly, but slowly and steadily we'll have something substantial.

All stable codebase needs to have releases(using git tags)

No releases yet.