libcrpm: Improving the Checkpoint Performance of NVM

libcrpm (Checkpoint-Recovery interface using Persistent Memory) is a new programming library to improve the checkpoint performance for applications running in NVM. Developers only need to follow the traditional crash-recovery programming paradigm.

libcrpm addresses two problems simultaneously that exist in the current NVM-based checkpoint-recovery libraries: (1) high write amplification when page-granularity incremental checkpointing is used, and (2) high persistence costs from excessive memory fence instructions when fine-grained undo-log or copy-on-write is used.

For more details, please refer to our paper:

@inproceedings{ren2022libcrpm,
  title={libcrpm: improving the checkpoint performance of NVM},
  author={Ren, Feng and Chen, Kang and Wu, Yongwei},
  booktitle={Proceedings of the 59th ACM/IEEE Design Automation Conference},
  pages={811--816},
  year={2022}
}

System Requirements

Optane DC persistent memory, configured as FSDAX mode and mounted to /mnt/pmem0.
Clang/LLVM 10.x, required by our instrumention tool. We built and verified libcrpm with clang 10.0.0, if you use other clang versions, you may have to modify codes to resolve interface incompatibility.
OpenMPI 4.x, required for building MPI support modules. Currently MPICH is not supported. mpicc should be located in the PATH directory.

Getting Started

Build kernel module

The flush directory includes a kernel module for wbinvd instruction invocation. This is extracted from InCLL. You should build and load this kernel module first, and this requires the root privilege. The linux-headers package should be installed.

cd flush
make
sudo insmod gf.ko
sudo chmod 777 /dev/global_flush

Build `libcrpm`

cd libcrpm
mkdir build; cd build; cmake ..; make -j
mkdir /mnt/pmem0/libcrpm

The path is hard-coded in the source code. You may have to replace it yourself.

Evaluate `libcrpm`

We provide test scripts for generating datasets and evaluating end-to-end performance of C++ STL data structures (map and unordered_map).

bash ../script/prepare-dataset.sh
bash ../script/perf-test.sh When each test completes, there is one line of output. The last two fields are execution time (nanoseconds) and throughput of this test (ops/sec).

You can also use the following command for run custom tests:

./tests/benchmark -d <dataset-path> -t <threads> 
                  -p <populated_records_before_execution> 
                  -i <checkpoint-interval>
                  -b <stl-map|stl-unordered-map> -e default -a default

As the starting point, we recommend you to read the tests directory for understanding the programming interface of libcrpm. It is no hard to transform your application to be recoverable.

Contact Authors

Feel free to contact the authors of libcrpm if you have trouble in installation and evaluation:

Feng Ren alogfans@gmail.com

alogfans/libcrpm