ursus-maritimus

Haskell port of the scala ursus-martimus repo

Ursus maritimus (A.K.A Polar Bear) is a tool for processing synthetically generated data in a streaming fashion.

Architecture:

+------------------------------------------------------------------------+
|                                                                        |
|                                                                        |
|   Data Generation Source             HTTP Endpoint                     |
|             +                              +                           |
|             |                              |                           |
|             |                              |                           |
|             v                              |                           |
|        JSON parser                         |  Query aggregated events  |
|             +                              |                           |
|             |                              |                           |
|             |                              |                           |
|             v         Publish to           v                           |
|       Raw event store +--------> Accumulated event store               |
|                                                                        |
|                                                                        |
+------------------------------------------------------------------------+

The architecture resembles the CQRS architecture where the write database is an append only log. After committing to the log, events are published to the read database which saves data in a more efficient manner for querying the data via the HTTP API.

Events in the accumulated event store are eventually consistent, in the sense that first events are written to the write store, they will be propogated to the read store and only then be visible to the user under the HTTP endpoint.

The following are prerequisites to running the application:

GHC 8.4.3
Stack
Data generation tool which generates events with the following JSON schema:

{
  "$schema": "http://json-schema.org/draft-04/schema#",
  "type": "object",
  "properties": {
    "event_type": {
      "type": "string"
    },
    "data": {
      "type": "string"
    },
    "timestamp": {
      "type": "integer"
    }
  },
  "required": [
    "event_type",
    "data",
    "timestamp"
  ]
}

How to run the application:

stack build
stack exec ursus-maritimus-exe [path to data generator]

Querying the data API

After running the service, the process starts an HTTP service which by default binds to "0.0.0.0:8080". If you'd like to change the port, this can be configured via the configuration.yaml, and a restart to the service.

Endpoints exposed: