/cbfs

ChuBao Filesystem (CBFS) is a unified platform for unstructured data storage

Primary LanguageGo

ChubaoFS

Overview

ChubaoFS is a unified distributed filesystem for small files and large files.

  • key-file data model, i.e., namespaceless, or hierachical namespace

  • unlimited scalability of both metadata and data

  • strong consistency

  • append-only or random writes

ChubaoFS has been built and deployed in production since 2013.

Architecture

metadata partition: inode range

data partition: two storage engines: tiny file chunks, extents; one replication protocol

It consists of several components:

  • the cluster master. single raft replication

  • metanode. multi-raft replication, a meta partition (inode range) per raft

  • datanode. de-clustering of data partitions, replicated via a simple but efficient consistent replication protocol

Interfaces

  • Go SDK
  • Java SDK
  • FUSE

Usage

  • Image Store

based on the namespace-less key-file interface, nginx integration

  • Object Store

integration with minio

  • Datacenter filesystem

run complex data workloads like databases on top of it

  • Container Persistent Volumes

integration with Kubernetes via CSI