JournalKeeper

中文版

JournalKeeper is a high performance, highly reliable, strong consistent distributed streaming data storage cluster. The JournalKeeper log consistency algorithm is derived from the RAFT consistency algorithm and has been extended and improved to be more suitable for very large scale clusters with better performance. It clearly divides the system into three parts: consistency log, state machine and storage, so that each part can conform to the principle of single responsibility. JournalKeeper clearly defines system boundaries, using a more systematic and structured description method to define this algorithm, making it easy to implement and apply it to engineering practice.

What is JournalKeeper？

JournalKeeper is a ...

Complete RAFT implementation

JournalKeeper implements and hides most of the complexity of RAFT, and provides a minimal API: JournalKeeper RAFT API. Take a look at journalkeeper-examples.

Distributed streaming data storage

JournalKeeper provides Partitioned Journal Store API (JK-PS API) which were used to store sreaming data. It can be used to store monitoring data, log data such as Binlog, and provides streaming data storage for stream computing. It can also be used to implement Pub / Sub systems. The following features are supported:

Elastic muti-partition journal
Strong consistency
Strict order
Transaction write
High reliability, high availability, and high performance

In process distributed coordinating service

JournalKeeper provides Journal Keeper Coordinating Service API(JK-CS API) can be used to coordinate distributed systems, and it is embedded in your business system, No extra deployment and maintenance is required. JournalKeeper can be used:

Store metadata, support KV and SQL
Leader election
Monitor node status
Distributed locks
GUID Generation

Features

The core of JournalKeeper is a high-performance RAFT library optimized for large-scale distributed clusters. It strictly implements all the features of The Raft consensus algorithm, including:

Replicated state machines
Log replication
Leader election
Cluster membership changes in Two-phase
Automatically create snapshots
Install snapshot
Log compaction

In addition, JournalKeeper also includes these useful features:

Multi-partition journal: The consistency log is divided into multiple logical partitions, and a separate state machine can be established for each partition to achieve parallel read and write on the premise of ensuring causal consistency
Parallel log replication: improve master-slave replication performance
Reading from follower: JournalKeeper supports clients to read data from followers under the premise of ensuring sequence consistency, improving cluster read performance;
Preferred Leader: Supports the designation of a node in the cluster as the preferred leader. When the preferred leader node is alive, the cluster's election strategy always selects the preferred leader node as the leader. This in Multi-Raft can effectively avoid the problem of leadership skew.
Observers: Observers only replicate logs from the cluster and generate status data. They can provide read services, but they do not participate in cluster elections. Joining observers can improve cluster read performance without affecting cluster write and election performance.