getong/openraft

rust raft with improvements

Openraft

Advanced Raft in 🦀 Rust using Tokio. Please ⭐ on github!

🪵🪵🪵 Raft is not yet good enough. This project intends to improve raft as the next-generation consensus protocol for distributed data storage systems (SQL, NoSQL, KV, Streaming, Graph ... or maybe something more exotic).

Currently, openraft is the consensus engine of meta-service cluster in databend.

• 🚀 Get started:

  • Openraft guide is the best place to get started,
  • Openraft docs for more in-depth details,
  • Openraft FAQ explains some common questions.

• 🙌 Questions?

  • Join the Discord channel,
  • or start a Discussion,
  • or join our wechat group: drmingdrmer.

• Openraft is derived from async-raft with several bugs fixed: Fixed bugs.

Status

• The features are almost complete for building an application.
• Performance: Supports 70,000 writes/sec for single writer, and 1,000,000 writes/sec for 256 writers. See: Performance
• Unit test coverage stands at 92%.
• The chaos test has not yet been completed, and further testing is needed to ensure the application's robustness and reliability.

API status

• Openraft API is not stable yet. Before 1.0.0, an upgrade may contain incompatible changes. Check our change-log. A commit message starts with a keyword to indicate the modification type of the commit:

  • DataChange: on-disk data types changes, which may require manual upgrade.
  • Change: if it introduces incompatible changes.
  • Feature: if it introduces compatible non-breaking new features.
  • Fix: if it just fixes a bug.

Versions

• Branch main has been under active development. The main branch is for the release-0.9.

• Branch release-0.8: Latest: ( v0.8.8 | Change log ); Upgrade guide: ⬆️ 0.7 to 0.8, ⬆️ 0.8.3 to 0.8.4; release-0.8 Won't accept new features but only bug fixes.

• Branch release-0.7: Latest: ( v0.7.6 | Change log ); Upgrade guide: ⬆️ 0.6 to 0.7; release-0.7 Won't accept new features but only bug fixes.

• Branch release-0.6: Latest: ( v0.6.8 | Change log ); release-0.6 won't accept new features but only bug fixes.

Roadmap

• 2022-10-31 Extended joint membership
• 2023-02-14 Minimize confliction rate when electing; See: Openraft Vote design; Or use standard raft mode with feature flag single-term-leader.
• 2023-04-26 Goal performance is 1,000,000 put/sec.
• Reduce the complexity of vote and pre-vote: get rid of pre-vote RPC;
• Support flexible quorum, e.g.: Hierarchical Quorums
• Consider introducing read-quorum and write-quorum, improve efficiency with a cluster with an even number of nodes.

Performance

The benchmark is focused on the Openraft framework itself and is run on a minimized store and network. This is NOT a real world application benchmark!!!

clients	put/s	ns/op
256	1,014,000	985
64	730,000	1,369
1	70,000	14,273

For benchmark detail, go to the ./cluster_benchmark folder.

Features

• Async and Event-Driven: Operates based on Raft events without reliance on periodic ticks, optimizing message batching for high throughput.
• Extensible Storage and Networking: Customizable via RaftLogStorage, RaftStateMachine and RaftNetwork traits, allowing flexibility in choosing storage and network solutions.
• Unified Raft API: Offers a single Raft type for creating and interacting with Raft tasks, with a straightforward API.
• Cluster Formation: Provides strategies for initial cluster setup as detailed in the cluster formation guide.
• Built-In Tracing Instrumentation: The codebase integrates tracing for logging and distributed tracing, with the option to set verbosity levels at compile time.

Functionality:

• ✅ Leader election: by policy or manually(trigger_elect()).
• ✅ Non-voter(learner) Role: refer to add_learner().
• ✅ Log Compaction(snapshot of state machine): by policy or manully(trigger_snapshot()).
• ✅ Snapshot replication.
• ✅ Dynamic Membership: using joint membership config change. Refer to dynamic membership
• ⛔️ Wont support: Single-step config change.
• ✅ Toggle heartbeat: enable_heartbeat.
• ✅ Toggle election: enable_elect.
• ✅ Purge log by policy or manually: purge_log().

Who use it

• yuyang0/rrqlite
• raymondshe/matchengine-raft

Contributing

Check out the CONTRIBUTING.md guide for more details on getting started with contributing to this project.

License

Openraft is licensed under the terms of the MIT License or the Apache License 2.0, at your choosing.