consensus-shipyard/mir

Mir-based SMR Performance

matejpavlovic opened this issue · 0 comments

matejpavlovic commented

Background: The Mir Project

The Mir project aims at developing a production-quality implementation of:

  • a general framework for easily implementing distributed protocols
  • a Byzantine fault-tolerant consensus protocol using the framework.

This implementation (in the Go language) will be available as the open-source mir library.
Our aim is integration with the Eudico Filecoin client,
specifically its ordering service.
Being framed as a library, however, Mir's goal is also to serve as a general-purpose high-performance BFT component of other projects.

The first fault-tolerant consensus protocol implemented within mir is a variant of ISS (next generation of the MirBFT consensus protocol, that appeared at EuroSys22 as State Machine Replication Scalability Made Simple). However, the framework is general enough to facilitate the implementation of other distributed protocols in the future. The framework is highly modular, allowing the developer of the distributed protocol to focus on the protocol logic, without having to care about network transport, storage, and cryptographic primitives, etc.
At the same time, the high modularity facilitates creating custom protocol implementations tailored to the consumer's needs. Mir hopes to be a building block of a next generation of distributed systems.

The mir library is still in development. Some of its components are implemented naively, putting an absolute priority on simplicity while sacrificing efficiency. This project aims at improving the performance of both mir itself and the protocols implemented on top of it.

Tasks

This project focuses on the performance of Trantor, total order broadcast (TOB) and state machine replication (SMR) implementation on top of Mir.
Being the first (and at the time of writing this also the only) implemented application on top of Mir, TOB/SMR is a natural candidate for optimizing end-to-end performance.

The tasks this project consists of are as follows.

  • Implement tools for profiling the performance of Mir and its TOB/SMR protocol suite (CPU usage, lock contention, ...).
  • Identify the most pressing performance bottlenecks.
  • Propose solutions to eliminate the identified performance bottlenecks.
  • Improve the performance of Mir and its Trantor implementation.

Expected Outcomes

  • The ability of Mir to produce performance profiles in an easily readable form (e.g., using pprof).
  • List of the narrowest performance bottlenecks in Mir and its TOB/SMR implementation.
  • List of bottleneck mitigation proposals.
  • Mitigation of selected bottlenecks.
  • Comparison of performance before / after having addressed the selected bottlenecks.

Required Skills

  • Experience in programming in the Go programming language
  • Basic knowledge of the usage of the Git version control system.
  • Understanding of distributed systems theory and consensus / TOB / SMR protocols
  • Proficiency in the English language

Contact Info