/bitkv-rs

A k/v storage engine inspired by bitcask written in Rust

Primary LanguageRustMIT LicenseMIT

Bitkv

github Build Codecov GitHub License

An efficient key-value storage engine, designed for fast reading and writing, which is inspired by Bitcask.

See Introduction, Installation and Usages for more details.

Introduction

Bitkv is a high-performance key-value storage system written in Rust. It leverages a log-structured design with an append-only write approach to deliver exceptional speed, reliability, and scalability.

Features

  • Efficient Key-Value Storage: Optimized for fast read and write operations with minimal overhead.
  • Diverse Index: Support BTree, Skiplist, BPlusTree index for multiple index strategies.
  • MemMap files for efficient I/O: To achieve rapid index reconstruction and enhance startup speeds
  • Low latency per item read or written: Benchmarks run on a Macintosh with Apple M1 Core:
    • Write latency: ~ 3.3 µs
    • Read latency: ~ 370 ns
  • Concurrency Support: fine-grained locking minimizes contentions.
  • WriteBatch transaction: commit a batch of writes to ensure atomicity.

Installation

To use bitkv in your project, add it as a dependency in your Cargo.toml file:

[dependencies]
bitkv-rs = "0.2.1"

Then, run cargo build to download and compile bitkv-rs and its dependencies.

For more detailed setup and compilation instructions, visit the Bitkv-rs GitHub repository.

Usages

Please see examples.

For detailed usage and API documentation, refer to the bitkv-rs Documentation.

TODO

  • Basic error handling
  • Merge files during compaction
  • Configurable compaction triggers and thresholds
  • WriteBactch transaction
  • Use mmap to read data file that on disk.
  • Optimize hintfile storage structure to support the memtable build faster
  • Http api server
  • Tests
  • Benchmark
  • Documentation
  • Increased use of flatbuffers option to support faster reading speed
  • Extend support for Redis Data Types

Contribution

Contributions to this project are welcome! If you find any issues or have suggestions for improvements, please raise an issue or submit a pull request.

License

Bitkv-rs is licensed under the [MIT license](https://github.com/example/bitkv-rs/blob/main/LICENSE-MIT), permitting use in both open source and private projects.
This license grants you the freedom to use bitkv-rs in your own projects, under the condition that the original license and copyright notice are included with any substantial portions of the Bitkv-rs software.