EventSourcing.JVM

Tutorial, practical samples and other resources about Event Sourcing in JVM. See also my similar repositories for .NET and NodeJS.

EventSourcing.JVM

Event Sourcing

What is Event Sourcing?

Event Sourcing is a design pattern in which results of business operations are stored as a series of events.

It is an alternative way to persist data. In contrast with state-oriented persistence that only keeps the latest version of the entity state, Event Sourcing stores each state change as a separate event.

Thanks for that, no business data is lost. Each operation results in the event stored in the databse. That enables extended auditing and diagnostics capabilities (both technically and business-wise). What's more, as events contains the business context, it allows wide business analysis and reporting.

In this repository I'm showing different aspects, patterns around Event Sourcing. From the basic to advanced practices.

What is Event?

Events, represent facts in the past. They carry information about something accomplished. It should be named in the past tense, e.g. "user added", "order confirmed". Events are not directed to a specific recipient - they're broadcasted information. It's like telling a story at a party. We hope that someone listens to us, but we may quickly realise that no one is paying attention.

Events:

are immutable: "What has been seen, cannot be unseen".
can be ignored but cannot be retracted (as you cannot change the past).
can be interpreted differently. The basketball match result is a fact. Winning team fans will interpret it positively. Losing team fans - not so much.

What is Stream?

Events are logically grouped into streams. In Event Sourcing, streams are the representation of the entities. All the entity state mutations ends up as the persisted events. Entity state is retrieved by reading all the stream events and applying them one by one in the order of appearance.

A stream should have a unique identifier representing the specific object. Each event has its own unique position within a stream. This position is usually represented by a numeric, incremental value. This number can be used to define the order of the events while retrieving the state. It can be also used to detect concurrency issues.

Event representation

Technically events are messages.

They may be represented, e.g. in JSON, Binary, XML format. Besides the data, they usually contain:

id: unique event identifier.
type: name of the event, e.g. "invoice issued".
stream id: object id for which event was registered (e.g. invoice id).
stream position (also named version, order of occurrence, etc.): the number used to decide the order of the event's occurrence for the specific object (stream).
timestamp: representing a time at which the event happened.
other metadata like correlation id, causation id, etc.

Sample event JSON can look like:

{
  "id": "e44f813c-1a2f-4747-aed5-086805c6450e",
  "type": "invoice-issued",
  "streamId": "INV/2021/11/01",
  "streamPosition": 1,
  "timestamp": "2021-11-01T00:05:32.000Z",

  "data":
  {
    "issuedTo": {
      "name": "Oscar the Grouch",
      "address": "123 Sesame Street",
    },
    "amount": 34.12,
    "number": "INV/2021/11/01",
    "issuedAt": "2021-11-01T00:05:32.000Z"
  },

  "metadata": 
  {
    "correlationId": "1fecc92e-3197-4191-b929-bd306e1110a4",
    "causationId": "c3cf07e8-9f2f-4c2d-a8e9-f8a612b4a7f1"
  }
}

Retrieving the current state from events

In Event Sourcing, the state is stored in events. Events are logically grouped into streams. Streams can be thought of as the entities' representation. Traditionally (e.g. in relational or document approach), each entity is stored as a separate record.

Id	IssuerName	IssuerAddress	Amount	Number	IssuedAt
e44f813c	Oscar the Grouch	123 Sesame Street	34.12	INV/2021/11/01	2021-11-01

In Event Sourcing, the entity is stored as the series of events that happened for this specific object, e.g. InvoiceInitiated, InvoiceIssued, InvoiceSent.

[
    {
        "id": "e44f813c-1a2f-4747-aed5-086805c6450e",
        "type": "invoice-initiated",
        "streamId": "INV/2021/11/01",
        "streamPosition": 1,
        "timestamp": "2021-11-01T00:05:32.000Z",

        "data":
        {
            "issuedTo": {
                "name": "Oscar the Grouch",
                "address": "123 Sesame Street",
            },
            "amount": 34.12,
            "number": "INV/2021/11/01",
            "initiatedAt": "2021-11-01T00:05:32.000Z"
        }
    },        
    {
        "id": "5421d67d-d0fe-4c4c-b232-ff284810fb59",
        "type": "invoice-issued",
        "streamId": "INV/2021/11/01",
        "streamPosition": 2,
        "timestamp": "2021-11-01T00:11:32.000Z",

        "data":
        {
            "issuedTo": "Cookie Monster",
            "issuedAt": "2021-11-01T00:11:32.000Z"
        }
    },        
    {
        "id": "637cfe0f-ed38-4595-8b17-2534cc706abf",
        "type": "invoice-sent",
        "streamId": "INV/2021/11/01",
        "streamPosition": 3,
        "timestamp": "2021-11-01T00:12:01.000Z",

        "data":
        {
            "sentVia": "email",
            "sentAt": "2021-11-01T00:12:01.000Z"
        }
    }
]

All of those events shares the stream id ("streamId": "INV/2021/11/01"), and have incremented stream position. We can get to conclusion that in Event Sourcing entity is represented by stream, so sequence of event correlated by the stream id ordered by stream position.

To get the current state of entity we need to perform the stream aggregation process. We're translating the set of events into a single entity. This can be done with the following the steps:

Read all events for the specific stream.
Order them ascending in the order of appearance (by the event's stream position).
Construct the empty object of the entity type (e.g. with default constructor).
Apply each event on the entity.

This process is called also stream aggregation or state rehydration.

Event Store

Event Sourcing is not related to any type of storage implementation. As long as it fulfils the assumptions, it can be implemented having any backing database (relational, document, etc.). The state has to be represented by the append-only log of events. The events are stored in chronological order, and new events are appended to the previous event. Event Stores are the databases' category explicitly designed for such purpose.

In the further samples, I'll use EventStoreDB. It's the battle-tested OSS database created and maintained by the Event Sourcing authorities. It supports many dev environments via gRPC clients, including JVM.

Videos

Practical introduction to Event Sourcing with Spring Boot and EventStoreDB

Let's build the worst Event Sourcing system!

The Light and The Dark Side of the Event-Driven Design

Conversation with Yves Lorphelin about CQRS

How to deal with privacy and GDPR in Event-Sourced systems

Support

Feel free to create an issue if you have any questions or request for more explanation or samples. I also take Pull Requests!

💖 If this repository helped you - I'd be more than happy if you join the group of my official supporters at:

👉 Github Sponsors

⭐ Star on GitHub or sharing with your friends will also help!

Introduction to Event Sourcing self-paced kit

Event Sourcing is perceived as a complex pattern. Some believe that it's like Nessie, everyone's heard about it, but rarely seen it. In fact, Event Sourcing is a pretty practical and straightforward concept. It helps build predictable applications closer to business. Nowadays, storage is cheap, and information is priceless. In Event Sourcing, no data is lost.

The workshop aims to build the knowledge of the general concept and its related patterns for the participants. The acquired knowledge will allow for the conscious design of architectural solutions and the analysis of associated risks.

The emphasis will be on a pragmatic understanding of architectures and applying it in practice using Marten and EventStoreDB.

Introduction to Event-Driven Architectures. Differences from the classical approach are foundations and terminology (event, event streams, command, query).
What is Event Sourcing, and how is it different from Event Streaming. Advantages and disadvantages.
Write model, data consistency guarantees on examples from Marten and EventStoreDB.
Various ways of handling business logic: Aggregates, Command Handlers, functional approach.
Projections, best practices and concerns for building read models from events on the examples from Marten and EventStoreDB.
Challenges in Event Sourcing and EDA: deliverability guarantees, sequence of event handling, idempotency, etc.
Saga, Choreography, Process Manager, distributed processes in practice.
Event Sourcing in the context of application architecture, integration with other approaches (CQRS, microservices, messaging, etc.).
Good and bad practices in event modelling.
Event Sourcing on production, evolution, events' schema versioning, etc.

You can do the workshop as a self-paced kit. That should give you a good foundation for starting your journey with Event Sourcing and learning tools like Marten and EventStoreDB. If you'd like to get full coverage with all nuances of the private workshop, feel free to contact me via email.

Read also more in my article Introduction to Event Sourcing - Self Paced Kit.

Exercises

Events definition.
Getting State from events.
Appending Events:
- EventStoreDB
- TODO: Axon Server
Getting State from events
- EventStoreDB
- TODO: Axon Server
Business logic:
- General
- EventStoreDB
- TODO: Axon Server
Optimistic Concurrency:
- EventStoreDB
- TODO: Axon Server
Projections:

Samples

See also fully working, real-world samples of Event Sourcing and CQRS applications in Samples folder.

Samples are using CQRS architecture. They're sliced based on the business modules and operations. Read more about the assumptions in "How to slice the codebase effectively?".

Event Sourcing with Spring Boot and EventStoreDB

Overview

Sample is showing basic Event Sourcing flow. It uses EventStoreDB for event storage and Spring Data JPA backed with PostgreSQL for read models.

The presented use case is Shopping Cart flow:

The customer may add a product to the shopping cart only after opening it.
When selecting and adding a product to the basket customer needs to provide the quantity chosen. The product price is calculated by the system based on the current price list.
The customer may remove a product with a given price from the cart.
The customer can confirm the shopping cart and start the order fulfilment process.
The customer may also cancel the shopping cart and reject all selected products.
After shopping cart confirmation or cancellation, the product can no longer be added or removed from the cart.

Technically it's modelled as Web API written in Spring Boot and Java 17.

There are two variations of those samples:

Main assumptions

explain basics of Event Sourcing, both from the write model (EventStoreDB) and read model part (PostgreSQL and Spring Data JPA),
present that you can join classical approach with Event Sourcing without making a massive revolution,
CQRS architecture sliced by business features, keeping code that changes together at the same place. Read more in How to slice the codebase effectively?,
clean, composable (pure) functions for command, events, projections, query handling, minimising the need for marker interfaces. Thanks to that testability and easier maintenance.
easy to use and self-explanatory fluent API for registering commands and projections with possible fallbacks,
registering everything into regular DI containers to integrate with other application services.
pushing the type/signature enforcement on edge, so when plugging to DI.

Prerequisites

For running the Event Store examples you need to have:

Java JDK 17 (or later) installed - https://www.oracle.com/java/technologies/downloads/.
Installed IntelliJ, Eclipse, VSCode or other preferred IDE.
Docker installed.

Tools used

EventStoreDB - Event Store
PostgreSQL - Read Models
Spring Boot - Web Application framework

Event Versioning

Shows how to handle basic event schema versioning scenarios using event and stream transformations (e.g. upcasting):

Uniqueness

Shows how to handle unique constraint checks in an event-sources system. Explains various techniques, like:

talking to business,
stream id design,
reservation pattern.

Distributed Processes

Shows how to handle distributed processes in Event Sourcing in practice. Explains various use cases, like:

batch processing,
saga vs process managers,
distributed processes in the single module and across boundaries,
internal vs external events,
compensating failures,
implementation of command and event bus in EventStoreDB.

Articles

Read also more on the Event Sourcing and CQRS topics in my blog posts:

qiozas/oskardudycz_EventSourcing.JVM