TopoLVM

TopoLVM is a CSI plugin using LVM for Kubernetes. It can be considered as a specific implementation of local persistent volumes using CSI and LVM.

The team presented the motivation and implementation of TopoLVM at KubeCon Europe 2020: https://kccnceu20.sched.com/event/ZerD

If you have any questions, please use discussions.

Project Status: Testing for production
Conformed CSI version: 1.5.0

Supported environments

Kubernetes: 1.25, 1.24, 1.23
Node OS: Linux with LVM2 (*1)
Filesystems: ext4, xfs, btrfs(experimental)
lvm version 2.02.163 or later (adds JSON output support)

*1 The host's Linux Kernel must be v4.9 or later which supports rmapbt and reflink, if you use xfs filesystem with an official docker image.

Features

Dynamic provisioning: Volumes are created dynamically when PersistentVolumeClaim objects are created.
Raw block volume: Volumes are available as block devices inside containers.
Topology: TopoLVM uses CSI topology feature to schedule Pod to Node where LVM volume exists.
Extended scheduler: TopoLVM extends the general Pod scheduler to prioritize Nodes having larger storage capacity.
Volume metrics: Usage stats are exported as Prometheus metrics from kubelet.
Volume Expansion: Volumes can be expanded by editing PersistentVolumeClaim objects.
Storage capacity tracking: You can enable Storage Capacity Tracking mode instead of using topolvm-scheduler.

Planned features

Snapshot: When we want it.

Programs

A diagram of components is available in docs/design.md.

This repository contains these programs:

topolvm-controller: CSI controller service.
topolvm-scheduler: A scheduler extender for TopoLVM.
topolvm-node: CSI node service.
lvmd: gRPC service to manage LVM volumes.

lvmd is a standalone program that should be run on Node OS as a systemd service. Other programs are packaged into a container image.

Getting started

A demonstration of TopoLVM running on kind (Kubernetes IN Docker) is available at example directory.

For production deployments, see deploy/README.md.

User manual is at docs/user-manual.md.

Deprecated: If you want to use TopoLVM on Rancher/RKE, see docs/deprecated/rancher.md.

Documentation

docs directory contains documents about designs and specifications.

Docker images

Docker images are available on ghcr.io
Please note that the images on quay.io are currently deprecated and will no longer be available in the future.

Deprecated features

TopoLVM currently uses Pod security policy, but it has been deprecated since Kubernetes 1.21, and will be deleted in Kubernetes 1.25. So it is also deprecated in TopoLVM.

qyz87/topolvm