CSI Resizer

The CSI external-resizer is a sidecar container that watches the Kubernetes API server for PersistentVolumeClaim updates and triggers ControllerExpandVolume operations against a CSI endpoint if user requested more storage on PersistentVolumeClaim object.

Overview

A storage provider that allows volume expansion after creation, may choose to implement volume expansion either via a control-plane CSI RPC call or via node CSI RPC call or both as a two step process. The external-resizer is an external-controller that watches Kubernetes API server for PersistentVolumeClaim modifications and triggers CSI calls for control-plane volume-expansion. More details can be found on - CSI Volume expansion

Compatibility

This information reflects the head of this branch.

Compatible with CSI Version	Container Image	Min K8s Version	Recommended K8s Version
CSI Spec v1.2.0	k8s.gcr.io/sig-storage/csi-resizer	1.16	1.16

Feature status

Currently all CSI volume expansion features are supported as Beta features by external-resizer.

Usage

It is necessary to create a new service account and give it enough privileges to run the external-resizer, see deploy/kubernetes/rbac.yaml. The resizer is then deployed as single Deployment as illustrated below:

kubectl create deploy/kubernetes/deployment.yaml

The external-resizer may run in the same pod with other external CSI controllers such as the external-attacher, external-snapshotter and/or external-provisioner.

Note that the external-resizer does not scale with more replicas. Only one external-resizer is elected as leader and running. The others are waiting for the leader to die. They re-elect a new active leader in ~15 seconds after death of the old leader.

Command line options

Recommended optional arguments

--csi-address <path to CSI socket>: This is the path to the CSI driver socket inside the pod that the external-resizer container will use to issue CSI operations (/run/csi/socket is used by default).
--leader-election: Enables leader election. This is mandatory when there are multiple replicas of the same external-resizer running for one CSI driver. Only one of them may be active (=leader). A new leader will be re-elected when current leader dies or becomes unresponsive for ~15 seconds.
--leader-election-namespace: Namespace where the leader election resource lives. Defaults to the pod namespace if not set.
--timeout <duration>: Timeout of all calls to CSI driver. It should be set to value that accommodates majority of ControllerExpandVolume calls. 10 seconds is used by default.
-kube-api-burst <int> : Burst to use while communicating with the kubernetes apiserver. Defaults to 10. (default 10).
-kube-api-qps <float> : QPS to use while communicating with the kubernetes apiserver. Defaults to 5.0. (default 5).
--retry-interval-start: The starting value of the exponential backoff for failures. 1 second is used by default.
--retry-interval-max: The exponential backoff maximum value. 5 minutes is used by default.
--workers <num>: Number of simultaneously running ControllerExpandVolume operations. Default value is 10.
--http-endpoint: The TCP network address where the HTTP server for diagnostics, including metrics and leader election health check, will listen (example: :8080 which corresponds to port 8080 on local host). The default is empty string, which means the server is disabled.
--metrics-path: The HTTP path where prometheus metrics will be exposed. Default is /metrics.
--handle-volume-inuse-error <true/false>: Enable or disable volume-in-use error handling in external-resizer. Defaults to true and resize-controller will watch for all pods in all namespaces to check if PVC being expanded is in-use by a pod or not before retrying volume expansion if CSI driver throws volume-in-use error. Setting this to false will cause external-resizer to ignore volume-in-use error and resize-controller will retry volume expansion even if volume is already in use by a pod and CSI driver does not support expansion of in-use volumes. If CSI driver being used supports online expansion, it might be desirable to set handle-volume-inuse-error to false - to save costs associated with watching all pods in the cluster.

Other recognized arguments

--kubeconfig <path>: Path to Kubernetes client configuration that the external-resizer uses to connect to Kubernetes API server. When omitted, default token provided by Kubernetes will be used. This option is useful only when the external-resizer does not run as a Kubernetes pod, e.g. for debugging. Either this or --master needs to be set if the external-resizer is being run out of cluster.
--master <url>: Master URL to build a client config from. When omitted, default token provided by Kubernetes will be used. This option is useful only when the external-resizer does not run as a Kubernetes pod, e.g. for debugging. Either this or --kubeconfig needs to be set if the external-resizer is being run out of cluster.
--metrics-address: (deprecated) The TCP network address where the prometheus metrics endpoint will run (example: :8080 which corresponds to port 8080 on local host). The default is empty string, which means metrics endpoint is disabled.
--version: Prints current external-resizer version and quits.
All glog / klog arguments are supported, such as -v <log level> or -alsologtostderr.

HTTP endpoint

The external-resizer optionally exposes an HTTP endpoint at address:port specified by --http-endpoint argument. When set, these two paths are exposed:

Metrics path, as set by --metrics-path argument (default is /metrics).
Leader election health check at /healthz/leader-election. It is recommended to run a liveness probe against this endpoint when leader election is used to kill external-resizer leader that fails to connect to the API server to renew its leadership. See kubernetes-csi/csi-lib-utils#66 for details.

Community, discussion, contribution, and support

Learn how to engage with the Kubernetes community on the community page.

You can reach the maintainers of this project at:

Code of conduct

Participation in the Kubernetes community is governed by the Kubernetes Code of Conduct.

sfowl/external-resizer