The NFS (Network File System) protocol is an integral part of cloud file sharing in enterprise IT environments. It’s this protocol that Linux-based devices rely on. Using NFS gives access to a wide range of users to share files and data concurrently. This is an important capability when using Kubernetes storage.
There’s a large number of use cases where NFS is used, including creating data lakes for building analytics solutions, database services, data archives, and more. NFS can be used with Kubernetes pods to manage persistent storage requirements and share data and files between containers and other pods. However, in enterprise-scale deployments, provisioning persistent NFS volumes that stateful applications need can be a challenge for admins to handle manually, and without costs getting out of control.
With Cloud Volumes ONTAP and Trident, Kubernetes NFS provisioning can be automatic and cost-efficient. In this post we’ll look at deploying Kubernetes with NFS volumes and give you some code examples on how to deploy persistent Kubernetes NFS volume storage using NetApp’s Trident and Cloud Volumes ONTAP.
Let’s first take a look at some of the reasons why you’d use shared storage in a Kubernetes cluster.
For starters, there’s an advantage to NFS over iSCSI because managing a large number of individual iSCSI storage allocations can add to administrative overhead. All those block-level persistent volume allocations can make storage management more difficult, especially when Kubernetes persistent volumes are statically allocated. When using block storage this way, your overall storage utilization may wind up being lower than planned, as persistent volumes are only used by a single pod, and that pod may never use up all of the storage that has been allocated for it. There’s another additional snag in that sharing data between pods can be difficult when using block-level persistent volumes.
Unless containers are deployed to the same pod, sharing data between containers can also be difficult. Deploying containers in this way, i.e. to the same pod, should only be done when it makes sense for the application, such as when the containers work together directly to fulfill some function. Putting containers into the same pod when they simply need to share data can lead to scalability problems as a pod will only be scheduled to a single node.
NFS solves these problems by allowing many hosts to mount the same file system at the same time, and for all those individual hosts to concurrently access the file system. Using NFS, storage volumes don’t need to be formatted according to an operating system file system, such as ext4—storage can simply be mounted and be put right to work. This makes it easier to attach storage to Kubernetes pods, reducing the administrative heavy lifting that usually comes with persistent storage. NFS storage volumes in Kubernetes can also be easily expanded without any client-side changes using Trident, which we’ll discuss in more detail in the next section.
NetApp Trident is a storage provisioner for Kubernetes that allows users to take advantage of NetApp storage services, both on-premises and in the cloud. Trident is a fully-supported, open-source solution that allows native Kubernetes manifests to be used to provision persistent volumes via Cloud Volumes ONTAP.
As a Kubernetes NFS volume provisioner, the benefits of using Trident include mounting persistent volumes as Read/Write Many, the ability to resize NFS persistent volumes dynamically, and the ability to create separate Kubernetes storage classes for use with different mount parameters and other requirements.
Persistent Volumes using NFS can be mounted as Read/Write Many, which allows multiple pods to mount the same volumes. Using native Kubernetes constructs means that storage can be allocated without the need for custom procedures or processes. Using the Read/Write Many access mode allows many persistent volumes claims to access the same persistent volume.
With block-level Kubernetes persistent volumes, Read/Write Once must be used, which means that there is a 1-to-1 relationship between persistent volume claims and persistent volumes.
If you’re looking to make a persistent volume claim (PVC), here’s how you would do that for a Kubernetes NFS volume:
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
name: nfs_share
spec:
accessModes:
- ReadWriteMany
resources:
requests:
storage: 100Gi
storageClassName: silver
Kubernetes supports editing persistent volume claims to request more storage space. With this capability, persistent volumes can be expanded without needing to re-create the pod that uses the storage. In order for that to happen, the underlying storage provisioner must support resize. Trident provides users with NFS-based persistent volumes that can be dynamically resized with all the back-end changes taken care of. This makes Trident a major value add.
By extending the support for resizing persistent volumes to NFS, Trident is a major value add.
Block-level storage, such as Amazon EBS, needs to have the file system extended when the underlying storage is expanded. Kubernetes will take care of this, but it will also require that the pod using the storage to be restarted. That will cause downtime. With NFS, no file system expansion is required, and the pod can continue to work without interruption.
You can read more about resizing NFS volumes with Trident here.
Storage classes allow users to control the type and configuration when provisioning volumes for use with Kubernetes. That extends to the dynamic Kubernetes NFS provisioning that Trident provides.
Separate storage classes can be created for different mount parameters and other requirements, for example using separate storage classes for NFSv3 and NFSv4.
Below is a storage class manifest for a Kubernetes NFS volume:
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: ontapnas
provisioner: netapp.io/trident
parameters:
backendType: ontap-nas
allowVolumeExpansion: true
The allowVolumeExpansion attribute controls whether the resizing of persistent volumes created using this storage class is allowed or not.
The storage class can be used to control the storage pool, or NetApp aggregate used to provision persistent volumes, e.g. a storage class named gold may be used to configure high performance back-end storage, whereas one called bronze may be configured for capacity storage.
Along with the ability to automatically provision persistent NFS volume storage dynamically that can be resized as needed, Cloud Volumes ONTAP has even more benefits for Kubernetes deployments. As an enterprise data management platform for use in AWS, Google Cloud, or Azure, it extends beyond cloud-specific native Kubernetes management services. It also allows persistent volumes to be used with iSCSI and SMB in addition to NFS. This makes Kubernetes file storage more flexible in large-scale enterprise deployments.
Plus, persistent storage with Cloud Volumes ONTAP benefits from data protection with NetApp Snapshot™ copies, instant, zero-capacity test copies with FlexClone®, and the data mobility and DR benefits that come with SnapMirror® data replication. Kubernetes deployments can also be protected with high availability configurations that provide redundancy across regions, ensuring business continuity. And Cloud Volumes ONTAP’s storage efficiencies can drastically lower your overall costs for Kubernetes persistent storage.
Using NFS for shared storage can be an attractive option because it can be easier to manage and more flexible than other storage formats, such as iSCSI. NetApp Trident and Cloud Volumes ONTAP make it easier to deploy and manage NFS for Kubernetes clusters in the cloud.
Using Trident as your Kubernetes NFS volume provisioner gives you some additional benefits that you won’t get with other provisioners, such as the ability to dynamically resize NFS persistent volumes. Combining that with Cloud Volumes ONTAP’s powerful data storage solutions, such as storage efficiency, volume cloning, and data replication, makes for an enterprise solution for Kubernetes and NFS.