Storage
Mobility
Protection
Analysis and control
TOPICShare
What Are Kubernetes Volumes?
A Kubernetes volume is a directory containing data, which can be accessed by containers in a Kubernetes pod. The location of the directory, the storage media that supports it, and its contents, depend on the specific type of volume being used.
Processes running within containers in a pod see a file system view composed of:
- A root file system that matches the content of the container image.
- Volumes mounted on the container (if defined). Each volume mounts on a specific path within the container file system.
Volumes are defined in the .spec.containers[*].volumeMounts field of the pod template. For each pod and each container image within the pod, you need to specify which volumes it will mount and on which paths (the paths can be different for each container).
There are a few types of volumes in Kubernetes. The most important are ephemeral volumes, which are stored locally on the Kubernetes node and are deleted when a pod restarts, and Kubernetes persistent volumes (PV) which retain data even after a pod shuts down.
In this article:
- Types of Kubernetes Volumes
- Plugins for Storing Kubernetes Volumes
- What Are Kubernetes volumeMounts?
- Creating a Kubernetes Volume by Deploying Pods
- Kubernetes Volume Management with NetApp Cloud Volumes ONTAP
5 Types of Kubernetes Volumes
Kubernetes supports various volumes, allowing each pod to use several volume types simultaneously. Ephemeral volumes are bound to the pod's lifetime, while persistent volumes can persist beyond the pod's lifetime. It means Kubernetes destroys ephemeral volumes once their pod no longer exists while keeping the data of persistent volumes.
Persistent Volumes
Kubernetes provides a PersistentVolume subsystem with an API that abstracts storage provisioning and consumption. It works with two API resources—PersistentVolume (PV) and PersistentVolumeClaim (PVC).
PersistentVolume (PV)
A PV is a storage resource located in the cluster. Administrators can manually provision PVs, and Kubernetes can use storage classes to dynamically provisioned PVs. Like volumes, PVs are plugins, but their lifecycle is independent of any pod using the PV.
A PV works as an API object that captures the details of the storage implementation, including iSCSI, NFS, and cloud provider storage systems. It works similarly to a node but offers storage resources rather than computing.
PersistentVolumeClaim (PVC)
A PVC is a storage request made by a user. It works similarly to a pod but consumes PV resources rather than node resources. A PVC can request specific storage resources, specifying size access modes such as ReadWriteOnce, ReadWriteMany, and ReadOnlyMany.
PVCs enable users to consume abstract storage resources, but users typically need PVs with varying properties for different problems. This is why cluster administrators often need to offer varying PVs that differ in more than size and access modes. They can do that without exposing users to implementation details through StorageClass resources.
Related content: Read our guide to Kubernetes PVC
Ephemeral Volumes
Ephemeral volumes do not store data persistently across restarts. These volumes are bound to the pod's lifetime, which means they are created and deleted along with the pod. It enables stopping and restarting pods without limiting them to the availability of persistent volume.
Ephemeral volumes are simple to deploy and manage. You can specify them inline in the pod spec. Ephemeral volumes are ideal for applications that do not require persistent storage, like caching services.
EmptyDir Volumes
An emptyDir volume is created when Kubernetes assigns a pod to a node. The lifespan of this volume is tied to a pod's lifecycle existing on that specific node. An emptyDir volume recreates when containers restart or crash. However, the data in this volume is deleted and lost when the pod is removed from the node, crashes, or dies.
After creating an emptyDir volume, you can declare the volume type name as a field in the pod manifest file. It shows under the volume property section with empty curly braces{} as the value. EmptyDir volumes are suitable mainly for temporary data storage. For example, you can use it for scratch space, like a disk-based merge.
You can store emptyDir volumes on the medium backing the node. For example, you can use network storage or SSD. Alternatively, you can set "memory" in the emptyDir.medium field and Kubernetes will mount a RAM-backed filesystem (tmpfs). Note that Kubernetes clears tmpfs on node reboot.
Kubernetes hostPath Volumes
A hostPath volume mounts a directory or file from the host node's filesystem into your pod.
Here are key use cases for hostPath volumes:
- Use a /var/lib/dockerhostPath—to run a container that requires access to Docker internals.
- Use a /sys hostPath—to run cAdvisor in a container.
- Allow a pod to specify a hostPath—to define if a certain hostPath should exist before the pod starts running and if it should be created.
- Specify a type for a hostPath volume—you can set this up in addition to the required path property.
HostPath volumes security
HostPath volumes pose many security risks. Avoid using these possible whenever possible. If you must use a HostPath volume, you should scope it only to the required directory or file and mount it as ReadOnly.
Here are key security risks:
- Exposed credentials—HostPaths can expose privileged system credentials or privileged APIs. Threat actors can use it to attack other parts of the cluster or for container escape.
- Root privileges—any directories or files created on the underlying hosts are writable only by root. If you want to write to a hostPath volume, you need to modify file permissions on the host or run the process as root inside a privileged container.
You can use the AdmissionPolicy to restrict HostPath access to certain directories. However, the policy is effective only if you require volumeMounts to use readOnly mounts.
Kubernetes Volumes ConfigMap
A ConfigMap enables injecting configuration data into pods. Data stored within a ConfigMap can be referenced in a configMap volume type and then consumed by containerized applications that run in a pod. You need to provide the name of the ConfigMap in the volume when referencing a ConfigMap. Kubernetes lets you customize the path for a specific entry in the ConfigMap.
Plugins for Storing Kubernetes Volumes
Kubernetes provides several storage plugins that provide access to storage devices deployed in a Kubernetes cluster. These are implemented using the StorageClass object.
Some of the main plugins currently supported by Kubernetes are GCEPersistentDisk, AWSElasticBlockStore, AzureDisk, Glusterfs, NFS, and iSCSI. For more details on these plugins, see the StorageClass documentation.
Let’s review two notable storage plugins in more detail.
NFS
Network File System (NFS) is a standard protocol for mounting storage devices as local drives. Kubernetes lets you mount an NFS volume as a local drive in a container. Because legacy code often accesses data via NFS, this plugin is very useful for migrating legacy workloads to Kubernetes.
There are two ways to access data via NFS and Kubernetes:
- Ephemeral NFS volumes—can be attached to existing NFS storage.
- PersistentVolumes with NFS—lets you set up managed resources on a cluster which are accessed via NFS.
CSI
The Container Storage Interface (CSI) is a standard interface that allows container orchestrators to expose storage systems to containers they manage. CSI allows storage vendors to create plugins that are “out of tree”, meaning they do not need to be checked into the Kubernetes code repository and are not shipped with Kubernetes.
There are many out of tree plugins based on CSI that are offered directly by storage vendors. The advent of CSI has made it much easier for storage technologies to support Kubernetes.
Related content: Read our guide to Container Storage Interface
What Are Kubernetes volumeMounts?
There are two steps involved in creating a volume and making it accessible to a pod:
- Declaring it in the spec:volumes property of the pod template, and then deploying the pod on some nodes
- Mounting the volume to a specific container using the spec:containers:<name>:volumeMounts property
These steps go hand in hand. When you create a volume, you must also mount it to a container, and you may not mount a volume without declaring it in the pod template.
Here is an example showing both creation and mounting of a volume in a pod template YAML configuration:
spec:
containers:
—name: my-app
image: nginx
volumeMounts:
—name: my-volume
mountPath: /app/config
volumes:
—name: my-volume
In this code:
- volumes (at the bottom) creates a volume named my-volume and attaches it to the pod
- volumeMounts defines how the volume is mounted, including the file path by which the volume will be available from inside the container
- It is essential that the same volume name is used in both places—the volume declaration and the volumeMounts property.
Creating a Kubernetes Volume by Deploying Pods
To create a Kubernetes volume, you deploy one or more pods that declare the volume. A common way to do this is via a Deployment object. Here is an example of a Deployment manifest that does the following:
- Deploys three pods, each with one NGINX container
- Declares an emptyDir volume
- Mounts the volume on each of the three containers at the root
apiVersion: apps/v1
kind: Deployment
metadata:
name: pods-with-volumes
spec:
replicas: 3
selector:
matchLabels:
app: demo
template:
metadata:
labels:
app: demo
spec:
containers:
—name: my-container
image: nginx:1.14.2
volumeMounts:
—mountPath: /
name: my-volume
volumes:
—name: my-volume
emptyDir: {}
Note that the name of this Deployment is defined in pods-with-volumes—this is how you refer to it in the Kubernetes environment.
Like we saw in the previous section, this Deployment object performs the two actions necessary to create a volume:
- The spec:template:volumes property declares the volume.
- The spec:containers:<my-container>:volumeMounts property mounts the volume to the containers.
Let’s assume this YAML file is saved under the name my-deployment.yaml. You can create the Deployment in your Kubernetes cluster using this command:
kubectl apply -f my-deployment.yamlTo verify the Deployment is running correctly with the expected volumes, run this command:
kubectl describe pods pods-with-volumesIf everything worked correctly, the output will show that each pod has a container named my-container with the requested mount point:
Mounts:
/ from my-volume (rw)The output will also show the volume running under each pod:
Volumes:
my-volume:
Type: EmptyDir (a temporary directory that shares a pod's lifetime)Kubernetes Volume Management with NetApp Cloud Volumes ONTAP
NetApp Cloud Volumes ONTAP, the leading enterprise-grade storage management solution, delivers secure, proven storage management services on AWS, Azure and Google Cloud. Cloud Volumes ONTAP capacity can scale into the petabytes, and it supports various use cases such as file services, databases, DevOps or any other enterprise workload, with a strong set of features including high availability, data protection, storage efficiencies, Kubernetes integration, and more.
In particular, Cloud Volumes ONTAP supports Kubernetes Persistent Volume provisioning and management requirements of containerized workloads.
Learn more about how Cloud Volumes ONTAP helps to address the challenges of containerized applications in these Kubernetes Workloads with Cloud Volumes ONTAP Case Studies.
