What should you do if you’re working in Kubernetes and need a test copy of an existing persistent Kubernetes storage volume, let’s say one you might use for software development testing?
In this blog we take you through the challenges of creating clones of persistent volumes, and how Cloud Volumes ONTAP and Astra Trident make cloning PVs for Kubernetes much easier thanks to NetApp zero-capacity FlexClone® data cloning.
Read on as we cover:
Creating test copies of production data is very often a requirement for DevOps engineers implementing CI/CD pipelines or setting up staging clusters for pre-production testing. Automated software test suites will often mutate the data that they access, which means that a fresh copy of the data is required in order to repeat the testing, with test cycles sometimes executed hundreds of times. Ensuring faster TTM (time to market) and the delivery of high-quality software relies upon developers working in parallel and executing tests as often as required.
But the normal way that Kubernetes users create such clones is to create all new persistent volumes and then restore a backup copy to that volume. Let’s look at why that option isn’t ideal:
It Takes Too Long
Normally, you’d need to allocate a new persistent volume (PV) and then restore a backup, a process that takes time and consumes storage space. This method of Kubernetes volume cloning is too slow to keep up with the pace of development and too expensive given the number of clones required in every dev/test cycle. With the size of the source data involved it just takes too much time to perform a restore. This leads to reduced developer productivity when multiple, up-to-date copies of PVs are required, which also have to be refreshed frequently.
It’s Not Scalable
Restoring data backups is not a scalable solution for creating test data sets because the amount of storage used to develop and support an application will be many times greater than the storage requirements of the production environment. That means a significant increase in cloud storage costs. That’s not something any developer wants to get blamed for.
It’s Not Cost- or Space-Efficient
The time taken to perform a restore and the storage space it uses can both be very wasteful, as usually most of the restored data remains unaffected by the software testing and must be recreated simply to bring the data to a consistent point or to get an up-to-date copy.
There is another option. Kubernetes volume cloning is a much easier process with NetApp Trident and Cloud Volumes ONTAP. With Cloud Volumes ONTAP you can instantly create highly-space-efficient, writable clones of existing PVs in direct response to persistent volume claims (PVCs).
NetApp Trident is a CSI-compliant provisioner for Kubernetes that performs dynamic storage provisioning using Cloud Volumes ONTAP for Google Cloud, AWS, or Azure, or other on-premises ONTAP storage systems. The major advantage of using Trident is that containerized applications can benefit from NetApp’s enterprise data management features. One of these features is data cloning which can help speed up TTM and reduce storage footprint and costs.
FlexClone data clones are instantly created, zero-capacity copies of existing data volumes. Since they are based on NetApp Snapshot™ technology, they can be spun up as soon as they’re needed, and only consume storage space for any changes that are made to the clone.
In contrast to restoring backup copies as data clones, not only is NetApp FlexClone able to instantly clone a source volume of any size, it also does so with zero storage penalty for any storage size required. NetApp Trident helps you take advantage of this for PVs in a Kubernetes cluster.
Read more about FlexClone data clones here.
This is how Kubernetes users can benefit: instead of always provisioning new PV in response to PVC, using Trident the user can create clones of existing volumes using K8s Volume Snapshot feature and create PVs from these snapshots. In the backend FlexClone technology is used to create PVs from the snapshots.
Before creating volume snapshots, an external snapshot controller and Custom Resource Definitions (CRD) should be created as prerequisites. Run the following commands to create volume snapshot CRDs:
kubectl apply -f
https://raw.githubusercontent.com/kubernetes-csi/external-snapshotter/release-3.0/client/config/crd/snapshot.storage.k8s.io_volumesnapshotclasses.yaml
kubectl apply -f
https://raw.githubusercontent.com/kubernetes-csi/external-snapshotter/release-3.0/client/config/crd/snapshot.storage.k8s.io_volumesnapshotcontents.yaml
kubectl apply -f
https://raw.githubusercontent.com/kubernetes-csi/external-snapshotter/release-3.0/client/config/crd/snapshot.storage.k8s.io_volumesnapshots.yaml
The following commands will create the snapshot-controller. Note that the manifests need to be edited to use the desired namespace.
kubectl apply -f
https://raw.githubusercontent.com/kubernetes-csi/external-snapshotter/release-3.0/deploy/kubernetes/snapshot-controller/rbac-snapshot-controller.yaml
kubectl apply -f
https://raw.githubusercontent.com/kubernetes-csi/external-snapshotter/release-3.0/deploy/kubernetes/snapshot-controller/setup-snapshot-controller.yaml
Now the VolumeSnapshotClass should be set up using the below code:
#Use apiVersion v1 for Kubernetes 1.20 and above. For Kubernetes 1.17 - 1.19, use apiVersion v1beta1.
apiVersion: snapshot.storage.k8s.io/v1
kind: VolumeSnapshotClass
metadata:
name: csi-snapclass
driver: csi.trident.netapp.io
deletionPolicy: Delete
The below code can be used to create the VolumeSnapshot from an existing PVC. Name of the PVC is pvctest and name of snapshot is pvctest-snapshot
#Use apiVersion v1 for Kubernetes 1.20 and above. For Kubernetes 1.17 - 1.19, use apiVersion v1beta1.
apiVersion: snapshot.storage.k8s.io/v1
kind: VolumeSnapshot
metadata:
name: pvctest-snapshot
spec:
volumeSnapshotClassName: csi-snapclass
source:
persistentVolumeClaimName: pvctest
Next step is to create the PVC from the snapshot, sample code for the same given below. Here the new PVC named pvcnew-from-snapshot is created from the snapshot datasource i.e., pvctest-snapshot
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: pvcnew-from-snapshot
spec:
accessModes:
- ReadWriteOnce
storageClassName: golden
resources:
requests:
storage: 3Gi
dataSource:
name: pvctest-snapshot
kind: VolumeSnapshot
apiGroup: snapshot.storage.k8s.io
The PVC can be attached to pods to provision PVs on demand. Sample yaml for attaching the PVC to a pod is given below:
apiVersion: v1
kind: Pod
metadata:
name: snapshot-pod
spec:
volumes:
- name: snp-volume
persistentVolumeClaim:
claimName: pvcnew-from-snapshot
containers:
- name: sample-container
image: nginx
ports:
- containerPort: 80
name: "http-server"
volumeMounts:
- mountPath: "/usr/share/nginx/html"
name: snp-volume
Volume-related operations are transparently handled by Trident. For example, when the VolumeSnapshot is deleted in K8s, the associated ONTAP snapshots are deleted by Trident in the backend. All snapshots of Trident volumes should be removed before the volumes can be deleted. If the snapshots are not deleted before deleting the Volume, the Trident status is set to “Deleting state”.
Note that trident uses K8s Volume snapshots, that require version 1.17 and above. It is GA from version 1.20. The snapshots created through FlexVol are space efficient. They are stable, scalable, recoverable and deliver high performance, best suited for mission critical workloads.
Persistent volumes provide storage for pods in a Kubernetes cluster, however, the provisioners used to create them do not typically provide mechanisms to stand up copies of existing volumes for testing. NetApp Trident is able to achieve this through a simple set of operations leveraging K8S volume snapshot feature and capabilities of Cloud Volumes ONTAP to instantly provision writable clones of existing PVs. Data cloning is just one of Cloud Volumes ONTAP’s cost-cutting features.