Kubernetes is the leading managed container orchestration platform preferred by customers deploying microservices-based architectures in the cloud. Azure Kubernetes Service offers Kubernetes as a managed service, where the container orchestration platform is handled by Azure, enabling customers to focus on the developments of applications. But how does AKS meet Kubernetes storage demands?
Containers are stateless, which means data is not stored locally—these containers depend on attached persistent volumes to handle the data lifecycle. This blog will walk through the steps required for provisioning persistent volumes and configuring them to be used by containers in Azure Kubernetes Service clusters.
By default, the storage associated with pods is deleted when the pod lifecycle ends. For stateful applications, however, storage is expected to be persistent so that the data can remain available every time the pods gets recreated in the cluster. Container persistent storage in a Kubernetes cluster is provisioned using the PersistentVolume subsystem, which provides PersistentVolume and PersistentVolumeClaim API resources.
The PersistentVolumeClaim requests for a specific storage class, say Azure disks or Azure Files, and the underlying storage resource gets provisioned. This PersistentVolumeClaim is referenced in the pod definition so that the provisioned storage is mounted to the pod. In this way the PersistentVolume is linked to the PersistentVolumeClaim whenever the provisioned storage is mounted to the pod requesting the resource.
There are four major steps in creating and attaching persistent storage using Azure disks in Azure Kubernetes Service.
Let’s explore this process in detail with sample configuration files.
Note: The following steps will assume that the AKS cluster is already provisioned and that the administrator has access to execute the commands listed in this blog. For details of provisioning AKS cluster and connecting to it, please refer to this resource document from Microsoft.
Customers can choose from premium or standard Azure disks to create the persistent volumes they attach to pods. These volumes are defined as premium or standard based on the Storage Class. There are two pre-built storage classes in AKS: default and managed-premium.
Default: This storage class uses the standard Azure storage which leverages HDDs in the backend.
Managed-premium: This storage class uses premium Azure storage which leverages high performance SSD and can be used for production workloads that need higher IOPS and throughput.
There are certain limitations associated with the built-in storage classes. For example, while using the built-in storage classes (i.e., default and managed-premium) the volume size cannot be resized after provisioning. AKS provides an option to edit the built-in storage classes to alter these default characteristics. One way to work around these defaults is to create your own custom storage classes.
In this section we’ll show you how users can create custom storage classes to suit their specific requirements.
1. Connect to the AKS cluster from your management tool of choice. For this demonstration we will be using Azure CL. From the Azure CLI, save the following manifest as storage-class.yaml
kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
name: managed-premium-custom
provisioner: kubernetes.io/Azure-disk
reclaimPolicy: Retain
allowVolumeExpansion: true
parameters:
storageaccounttype: Premium_LRS
kind: Managed
This file can be used to create a storage class that uses Premium_LRS managed disks. The reclaimPolicy is set to retain so that the persistent volume is not deleted even if the pod to which it is attached gets deleted. With the parameter allowVolumeExpansion set to true, the volume can now be resized even after provisioning.
2. Apply the yaml file using the following command
$ kubectl apply -f storage-class.yaml
Upon successful execution, you will get a message that the storage class has been created.
The next step is to create a persistent volume claim (PVC), which uses the storage class defined in the above to provision an Azure disk as a persistent volume.
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: Azure-managed-disk
spec:
accessModes:
- ReadWriteOnce
storageClassName: managed-premium-custom
resources:
requests:
storage: 10Gi
The PVC will request for an Azure disk of 10 GB with accessmode as ReadWriteOnce. That means only one node can mount the volume as read-write.
2. To create the volume itself, run the following command:
$ kubectl apply -f Azure-pvc.yaml
On successful execution you will see a message that the persistent volume claim has been created.
After creation, the provisioned volume can also be seen from the Azure portal. Browse to the resource group where the AKS nodes are created to find the newly provisioned disk listed there.
You can also run the following command to list the pvc status. It will be listed as “bound” to the created persistent volume:
After creating the persistent volume claim and the Azure disk, it can be attached to a new pod by referencing the name of the persistent volume claim.
1. Create file Nginx.yaml with the following content
kind: Pod
apiVersion: v1
metadata:
name: nginx
spec:
containers:
- name: nginxfrontend
image: nginx
volumeMounts:
- mountPath: "/mnt/Azure"
name: volume
volumes:
- name: volume
persistentVolumeClaim:
claimName: Azure-managed-disk
The volume is mounted at /mnt/Azure as specified by the mountpath parameter
Note that the image for the container is taken from a public registry. Alternatively, it can also be downloaded from the Azure container registry.
2. Next, create the pod using the following command:
$ kubectl apply -f Nginx.yaml
When successfully executed, the pod will be created.
3. Verify that the persistent volume is attached to the pod by using the following command:
$kubectl describe pod nginx
You can see that the volume is listed and and uses the claim name “Azure-managed-disk.”
4. To connect to the running Azure container instance, use the following command and run the df command to view the volumes:
$kubectl exec -it nginx -- /bin/bash
#df
While Azure disks can provide AKS with persistent volumes for use in containers, NetApp Cloud Volumes ONTAP helps Kubernetes users leverage enterprise class data management features not otherwise available in Azure.
Cloud Volumes ONTAP offers file and block level storage service for workloads in Azure, serving data over NFS, SMB/CIFS, and iSCSI protocols. Cloud Volumes ONTAP can be used for provisioning persistent volumes in Kubernetes clusters through NetApp Trident, which is an open source project maintained by NetApp designed to help customers with the container storage requirements, whether for persistent storage for Docker, Kubernetes, OpenShift, or any other container platform.
NetApp Trident integrates with Kubernetes persistent volume subsystem to provision and manage volumes using Cloud Volumes ONTAP that can be mounted as persistent volumes in containers running on AKS.
Cloud Volumes ONTAP uses Azure storage in the backend, but provides much more value add to connected workloads through data management capabilities for persistent volumes and containerized workloads, including
Persistent volume configuration is an integral part of container deployment in AKS, as stateful sets need the data to be stored outside of the pod lifecycle. Along with options like Azure disks and Azure files, Cloud Volumes ONTAP can also be used to meet the requirements of such workloads. Cloud Volumes ONTAP helps extends the advantages of ONTAP technology to containers by offering highly available, durable and reliable persistent storage for Azure microservices deployed in AKS.
To read more about Kubernetes storage, check out the rest of the blogs on our hub: