/znode

A new "fast" node implementation for "Z" currencies (ZEC, ZEN)

Primary LanguageC++Apache License 2.0Apache-2.0

Znode - C++20 Bitcoin like node implementation

Table of Contents

About Znode

Znode is mostly a greenfield C++20 (or better) implementation for a bitcoin-like node. The "Z" depicts most of the inspiration is taken from zcashd and zend.
Main purpose of this work is to chase the efficiency frontier leveraging, at maximum extent possible, parallel programming and multitasking while, at the same time, maintaining high software quality standards, coherent coding styles, and maximum readability. The main architectural source of inspiration is from Silkworm: a project I contributed to in the past. Znode uses libmdbx as the internal database engine. This project is under active development and hasn't reached the alpha phase yet. For this reason there are no releases so far.

Project goals

Main target for this work is the ZEN cryptocurrency network: their main chain node implementation still relies (Nov 2023) on an outdated version of Bitcoin's Core which can, and should, be replaced with a more modern, more efficient and faster implementation.

  • Reduce memory footprint
  • Increase execution concurrency
  • Reduce synchronization times by at least 65% (at Nov 2023 is roughly 18 hours)
  • Reduce block propagation times across the network
  • Increase throughput
  • Implement pluggable consensus rules

Please be aware that although this list might not be exhaustive, it is not a wishlist either. In any case this is a personal project, and I'm not bound to any deadline or roadmap.
Should things change in the future (e.g. benevolent funding), I'll update this list accordingly.

Disclaimer

This project is not affiliated with, nor endorsed or funded by either zcashd or zend authors / owners / maintainers. I am a former contractor for the company owning the zend project, but I'm not anymore: this project is not meant to be adversarial to them in any way or form. What I'm working on here is "a" way to improve efficiency, but it does not pretend to be "the only" way. Should this get to a stable release sometime in the future, it'd mean I have added something to the "code diversity" which should be enforced in any cryptocurrency ecosystem claiming to be permission-less.
Nevertheless this is a personal project, so far, which I'm developing in my spare time. Should you find it useful, if nothing else for hints about your development, feel free to use it, as a whole or in pieces, as you wish in compliance with the license.

I'm not responsible for any damage, loss of data, loss of money, loss of time, loss of sleep, loss of hair, loss of friends, loss of family, loss of pets, loss of anything else you may incur by using or engaging with this software.

Obtaining Source Code

To obtain the source code for the first time you need to install Git on your computer and

$ git clone --recurse-submodules https://github.com/AndreaLanfranchi/znode.git
$ cd znode

We use some git submodules (which may eventually have their own submodules) : so after you've updated to the latest code with git pull remember to also update submodules with

$ git submodule update --init --recursive

Building on Linux & macOS

Ensure you have the following requirements installed :

  • C++20 compatible compiler and its support libraries: GCC >= 13 or Clang >= 13 (see here the compatibility matrix)
  • CMake >= 3.16.12
  • Perl >= 5.x

Once the prerequisites are installed boostrap cmake by running

$ mkdir build
$ cd build
$ cmake [-DCMAKE_BUILD_TYPE="[Debug|Release|RelWithDebInfo|MinSizeRel]"] ..

On the very first run of this command the toolchain will download and build additional components like, for example, the Boost Library. This operation may take some time depending on the capabilities of your hardware and your internet connection. Please be patient.

If you're on linux and have both GCC and Clang installed CMAKE will use GCC by default. Should you want to force the build using clang simply export the following variables before invoking cmake

$ export CC=/usr/bin/clang
$ export CXX=/usr/bin/clang++
$ cmake [-DCMAKE_BUILD_TYPE="[Debug|Release|RelWithDebInfo|MinSizeRel]"] ..

Important Should you already have built with GCC earlier remember do erase the build directory and re-create it.

Additional CMAKE options (specify with -D<OPTION_NAME[:type]>=<value>):

OPTION_NAME Description Default
BUILD_CORE_ONLY Only build Znode Core components OFF
BUILD_CLANG_COVERAGE Clang (only) instrumentation for code coverage OFF
BUILD_SANITIZE Build instrumentation for sanitizers OFF
BUILD_TESTS Build unit / consensus siphash_tests ON

Then run the build itself

$ make -j

Note about parallel builds using -j: if not specified the exact number of parallel tasks, the compiler will spawn as many as the cores available. That may cause OOM errors if the build is executed on a host with a large number of cores but a relatively small amount of RAM. To work around this, either specify -jn where n is the number of parallel tasks you want to allow or remove -j completely. Typically, for Znode each compiler job requires up to 4GB of RAM. So if, for example, your total RAM is 16GB then -j4 should be OK, while -j8 is probably not. It also means that you need a machine with at least 4GB RAM to compile Znode.

Now you can run the unit siphash_tests (if you have chosen to build them). There's one for core and one for node.

$ ./cmd/test/znode-core-test
$ ./cmd/test/znode-main-test

Along with siphash_tests also benchmarks are built. If you want to play with them run

$ ./cmd/benckmark/znode-core-benchmarks
$ ./cmd/benchmark/znode-infra-benchmarks

Building on Windows

*Note! Native Windows builds are maintained for compatibility/portability reasons. However, due to the lack of 128-bit integers support by MSVC, execution performance may be slightly impacted when compared to nix builds.

To be able to build on Windows you have to ensure the following requirements are installed

If you're willing to use Visual Studio (Community Edition is fine) as your primary IDE then the build tools are already included in the setup package (still you have to ensure the required components are installed). Alternatively you can use VSCode or CLion

For Visual Studio setups follow these instructions:

  • Ensure you've cloned the project just as described here
  • Open Visual Studio and select File -> Open -> Cmake...
  • Browse the folder where you have cloned this repository and select the file CMakeLists.txt
  • Let CMake cache generation complete : on first run this may take several minutes, depending on your hardware and internet connection capabilities, as it will download and build additional components like, for example, Boost library.
  • Solution explorer shows the project tree.
  • To build simply CTRL+Shift+B
  • Build files, libraries and executable binaries are written to "${projectDir}\build\ If you want to change this path simply edit CMakeSettings.json file and choose an output directory which does not pollute the source directory tree (e.g. %USERPROFILE%\.cmake-builds\${projectName}\)

We've deliberately chosen to force cmake generator to Visual Studio 17 2022 Win64 even if it might result being slower than Ninja: fact is Boost libraries fail to build properly on MSVC toolchain using Ninja generator.

Memory compression on Windows 10/11

Windows 10/11 provide a memory compression feature which makes available more RAM than what physically mounted at cost of extra CPU cycles to compress/decompress while accessing data. As MDBX is a memory mapped file this feature may impact overall performances. Is advisable to have memory compression off. Use the following steps to detect/enable/disable memory compression:

  • Open a PowerShell prompt with Admin privileges
  • Run Get-MMAgent (check whether memory compression is enabled)
  • To disable memory compression : Disable-MMAgent -mc and reboot
  • To enable memory compression : Enable-MMAgent -mc and reboot

Style guide

We use standard C++20 programming language. We adhere to Google's C++ Style Guide with the following differences:

  • C++20 rather than C++17
  • snake_case for source and header file names (ISO)
  • snake_case() for function and variable names (ISO)
  • member_variable_ names MUST have underscore suffix
  • prefixing variable names with initial abbreviation of underlying type (e.g vector<char> vChar{}) is highly discouraged
  • classes and struct names MUST be in Pascal Case (class FancyFoo)
  • prefer using instead of typedef
  • .cpp/.hpp file extensions for C++ : .c/.h are reserved for C
  • using namespace foo is allowed into source files but not inside headers unless limited in a reduced scope (i.e. inside a template class or function)
  • Exceptions are allowed only outside the core library
  • User-defined literals are allowed
  • Maximum line length is 120, indentation is 4 spaces - see .clang-format
  • Use #pragma once in the headers instead of the classic #ifndef guards.
  • Comments MUST adhere to Doxygen formats (excluding inline ones)
  • Avoid implicit conversions (e.g. int to bool)
  • Avoid auto when the type is not immediately obvious (e.g. auto foo = get_foo())
  • Use auto when the type is immediately obvious (e.g. auto foo = std::make_unique<Foo>())
  • Prefer if(ptr == nullptr) over if(!ptr) for immediate clarity
  • Prefer usage of monadic return values over simple bool ( e.g. std::optional<T>, std::variant<T, E>, std::expected<T, E>)

Developers willing to contribute are strongly encouraged to take a thorough read of this best practices about naming and layout

Don't reinvent the wheel !

While it is often tempting to write your own implementation of a well known algorithm or data structure, we strongly encourage you to use the ones provided by the standard library or by one of the following well-known and widely used libraries:

If you can't find what you need there, maybe a quick search on GitHub might help finding a good library which already has everything you need. A good starting point might be this awesome list of C++ libraries.

Consider writing your own implementation as last resort and only if you can't find anything suitable.