Azure Key Vault provider for Secrets Store CSI driver allows you to get secret contents stored in an Azure Key Vault instance and use the Secrets Store CSI driver interface to mount them into Kubernetes pods.
- Mounts secrets/keys/certs on pod start using a CSI volume
- Supports mounting multiple secrets store objects as a single volume
- Supports pod identity to restrict access with specific identities
- Supports pod portability with the SecretProviderClass CRD
- Supports windows containers (Kubernetes version v1.18+)
- Supports sync with Kubernetes Secrets (Secrets Store CSI Driver v0.0.10+)
- Supports multiple secrets stores providers in the same cluster.
- Demo
- Usage
- Azure Key Vault Provider Features
- Troubleshooting
- Contributing
- Testing
- Other Azure Clouds
- Support
This guide will walk you through the steps to configure and run the Azure Key Vault provider for Secrets Store CSI driver on Kubernetes.
Prerequisites
Recommended Kubernetes version:
- For Linux - v1.16.0+
- For Windows - v1.18.0+
For Kubernetes version 1.15 and below, please use Azure Keyvault Flexvolume
Deployment using Helm
Follow this guide to install the Secrets Store CSI driver and the Azure Key Vault provider using Helm.
Alternatively, follow this guide to install using deployment yamls.
In addition, if you are using Secrets Store CSI Driver and the Azure Keyvault Provider in a cluster with pod security policy enabled, review and create the following policy that enables the spec required for Secrets Store CSI Driver and the Azure Keyvault Provider to work:
kubectl apply -f https://raw.githubusercontent.com/Azure/secrets-store-csi-driver-provider-azure/master/deployment/pod-security-policy.yamlIn addition to a Kubernetes cluster, you will need an Azure Key Vault resource with secret content to access. Follow this quickstart tutorial to deploy an Azure Key Vault and add an example secret to it.
Review the settings you desire for your Key Vault, such as what resources (Azure VMs, Azure Resource Manager, etc.) and what kind of network endpoints can access secrets in it.
Take note of the following properties for use in the next section:
- Name of secret object in Key Vault
- Secret content type (secret, key, cert)
- Name of Key Vault resource
- Resource group the Key Vault resides within
- Azure Subscription ID the Key Vault was provisioned with
- Azure Tenant ID the Subscription ID belongs to
Create a SecretProviderClass custom resource to provide provider-specific parameters for the Secrets Store CSI driver. In this example, use an existing Azure Key Vault or the Azure Key Vault resource created previously.
NOTE: The
SecretProviderClasshas to be in the same namespace as the pod referencing it.
Update this sample deployment to create a SecretProviderClass resource to provide Azure-specific parameters for the Secrets Store CSI driver.
To provide identity to access key vault, refer to the following section.
apiVersion: secrets-store.csi.x-k8s.io/v1alpha1
kind: SecretProviderClass
metadata:
name: azure-kvname
spec:
provider: azure
parameters:
usePodIdentity: "false" # [OPTIONAL for Azure] if not provided, will default to "false"
useVMManagedIdentity: "false" # [OPTIONAL available for version > 0.0.4] if not provided, will default to "false"
userAssignedIdentityID: "client_id" # [OPTIONAL available for version > 0.0.4] use the client id to specify which user assigned managed identity to use. If using a user assigned identity as the VM's managed identity, then specify the identity's client id. If empty, then defaults to use the system assigned identity on the VM
keyvaultName: "kvname" # the name of the KeyVault
cloudName: "" # [OPTIONAL available for version > 0.0.4] if not provided, azure environment will default to AzurePublicCloud
cloudEnvFileName: "" # [OPTIONAL available for version > 0.0.7] use to define path to file for populating azure environment
objects: |
array:
- |
objectName: secret1
objectAlias: SECRET_1 # [OPTIONAL available for version > 0.0.4] object alias
objectType: secret # object types: secret, key or cert. For Key Vault certificates, refer to https://github.com/Azure/secrets-store-csi-driver-provider-azure/blob/master/docs/getting-certs-and-keys.md for the object type to use
objectVersion: "" # [OPTIONAL] object versions, default to latest if empty
- |
objectName: key1
objectAlias: "" # If provided then it has to be referenced in [secretObjects].[objectName] to sync with Kubernetes secrets
objectType: key
objectVersion: ""
resourceGroup: "rg1" # [REQUIRED for version < 0.0.4] the resource group of the KeyVault
subscriptionId: "subid" # [REQUIRED for version < 0.0.4] the subscription ID of the KeyVault
tenantId: "tid" # the tenant ID of the KeyVault
| Name | Required | Description | Default Value |
|---|---|---|---|
| provider | yes | specify name of the provider | "" |
| usePodIdentity | no | specify access mode: service principal or pod identity | "false" |
| useVMManagedIdentity | no | [available for version > 0.0.4] specify access mode to enable use of VM's managed identity | "false" |
| userAssignedIdentityID | no | [available for version > 0.0.4] the user assigned identity ID is required for VMSS User Assigned Managed Identity mode | "" |
| keyvaultName | yes | name of a Key Vault instance | "" |
| cloudName | no | [available for version > 0.0.4] name of the azure cloud based on azure go sdk (AzurePublicCloud, AzureUSGovernmentCloud, AzureChinaCloud, AzureGermanCloud, AzureStackCloud) | "" |
| cloudEnvFileName | no | [available for version > 0.0.7] path to the file to be used while populating the Azure Environment (required if target cloud is AzureStackCloud). More details here. | "" |
| objects | yes | a string of arrays of strings | "" |
| objectName | yes | name of a Key Vault object | "" |
| objectAlias | no | [available for version > 0.0.4] specify the filename of the object when written to disk - defaults to objectName if not provided | "" |
| objectType | yes | type of a Key Vault object: secret, key or cert. For Key Vault certificates, refer to doc for the object type to use. |
"" |
| objectVersion | no | version of a Key Vault object, if not provided, will use latest | "" |
| objectFormat | no | [available for version > 0.0.7] the format of the Azure Key Vault object, supported types are pem and pfx. objectFormat: pfx is only supported with objectType: secret and PKCS12 or ECC certificates |
"pem" |
| objectEncoding | no | [available for version > 0.0.8] the encoding of the Azure Key Vault secret object, supported types are utf-8, hex and base64. This option is supported only with objectType: secret |
"utf-8" |
| resourceGroup | no | [required for version < 0.0.4] name of resource group containing key vault instance | "" |
| subscriptionId | no | [required for version < 0.0.4] subscription ID containing key vault instance | "" |
| tenantId | yes | tenant ID containing key vault instance | "" |
The Azure Key Vault Provider offers four modes for accessing a Key Vault instance:
- Service Principal
- Pod Identity
- VMSS User Assigned Managed Identity
- VMSS System Assigned Managed Identity
To ensure your application is using the Secrets Store CSI driver, update your deployment yaml to use the secrets-store.csi.k8s.io driver and reference the SecretProviderClass resource created in the previous step.
Update your linux deployment yaml or windows deployment yaml to use the Secrets Store CSI driver and reference the SecretProviderClass resource created in the previous step.
volumes:
- name: secrets-store-inline
csi:
driver: secrets-store.csi.k8s.io
readOnly: true
volumeAttributes:
secretProviderClass: "azure-kvname"-
Deploy the SecretProviderClass yaml created previously. For example:
kubectl apply -f ./examples/v1alpha1_secretproviderclass.yaml -
Deploy the application yaml created previously. For example:
kubectl apply -f ./examples/nginx-pod-inline-volume-service-principal.yaml
To validate, once the pod is started, you should see the new mounted content at the volume path specified in your deployment yaml.
## show secrets held in secrets-store
kubectl exec -it nginx-secrets-store-inline ls /mnt/secrets-store/
## print a test secret held in secrets-store
kubectl exec -it nginx-secrets-store-inline cat /mnt/secrets-store/secret1
On pod start and restart, the driver will call the Azure provider binary to retrieve the secret content from the Azure Key Vault instance you have specified in the SecretProviderClass custom resource. Then the content will be mounted to the container's file system.
To validate, once the pod is started, you should see the new mounted content at the volume path specified in your deployment yaml.
kubectl exec -it nginx-secrets-store-inline ls /mnt/secrets-store/
fooIn some cases, you may want to create a Kubernetes Secret to mirror the mounted content. Use the optional secretObjects field to define the desired state of the synced Kubernetes secret objects.
NOTE: Make sure the
objectNameinsecretObjectsmatches the name of the mounted content. This could be the object name or the object alias.
A SecretProviderClass custom resource should have the following components:
apiVersion: secrets-store.csi.x-k8s.io/v1alpha1
kind: SecretProviderClass
metadata:
name: my-provider
spec:
provider: azure
secretObjects: # [OPTIONAL] SecretObject defines the desired state of synced K8s secret objects
- data:
- key: username # data field to populate
objectName: foo1 # name of the mounted content to sync. this could be the object name or the object alias
secretName: foosecret # name of the Kubernetes Secret object
type: Opaque # type of the Kubernetes Secret object e.g. Opaque, kubernetes.io/tlsNOTE: Here is the list of supported Kubernetes Secret types:
Opaque,kubernetes.io/basic-auth,bootstrap.kubernetes.io/token,kubernetes.io/dockerconfigjson,kubernetes.io/dockercfg,kubernetes.io/ssh-auth,kubernetes.io/service-account-token,kubernetes.io/tls.
- Here is a sample
SecretProviderClasscustom resource that syncs a secret from Azure Key Vault to a Kubernetes secret. - To view an example of type
kubernetes.io/tls, refer to the ingress-controller-tls sample
Once the secret is created, you may wish to set an ENV VAR in your deployment to reference the new Kubernetes secret.
spec:
containers:
- image: nginx
name: nginx
env:
- name: SECRET_USERNAME
valueFrom:
secretKeyRef:
name: foosecret
key: usernameHere is a sample deployment yaml that creates an ENV VAR from the synced Kubernetes secret.
You can setup the Secrets Store CSI Driver to periodically update the pod mount and Kubernetes Secret with the latest content from external secrets-store. Refer to doc for steps on enabling auto rotation.
NOTE The CSI driver does not restart the application pods. It only handles updating the pod mount and Kubernetes secret similar to how Kubernetes handles updates to Kubernetes secret mounted as volumes.
To troubleshoot issues with the csi driver and the provider, you can look at logs from the secrets-store container of the csi driver pod running on the same node as your application pod:
kubectl get pod -o wide
# find the secrets store csi driver pod running on the same node as your application pod
kubectl logs csi-secrets-store-secrets-store-csi-driver-7x44t secrets-storeFor 0.0.9+ the provider logs are available in the provider pods. To troubleshoot issues with the provider, you can look at logs from the provider pod running on the same node as your application pod
kubectl get pod -o wide
# find the csi-secrets-store-provider-azure pod running on the same node as your application pod
kubectl logs csi-csi-secrets-store-provider-azure-lmx6pPlease refer to CONTRIBUTING.md for more information.
For documentation on how to locally test the Secrets Store CSI Driver Provider for Azure, please refer to this guide
For documentation on how to pull secret content from air-gapped and/or on-prem clouds (such as Azure Stack Hub), please refer to this guide.
Azure Key Vault Provider for Secrets Store CSI Driver is an open source project that is not covered by the Microsoft Azure support policy. Please search open issues here, and if your issue isn't already represented please open a new one. The project maintainers will respond to the best of their abilities.
This demo created by Houssem Dellai is using AAD Pod Identity and Secret Store CSI provider for Key Vault to retrieve database login and password from Azure Key Vault. Watch it here.
This demo created by Nilesh Gule is using VMSS Managed Identity and Secret Store CSI provider for Key Vault to retrieve RabbitMQ related secret from Azure Key Vault. These secrets are synced with Kubernetes Secret object and then injected into deployment as ENV variables. Integrate Azure Key Vault (AKV) with Azure Kubernetes Service (AKS) in 5 easy steps
