Storage
Mobility
Protection
Analysis and control
TOPICShare
In recent years, software developers and DevOps engineers have benefited from encapsulating applications into lightweight, independent units called containers. Kubernetes storage takes container deployment to a whole new level by providing a robust solution for managing and scaling containers and containerized applications and workloads across a cluster of machines.
In this introduction to Kubernetes, the first part of our series on the subject, we’ll help you learn Kubernetes starting with the basics: the history of containers and container orchestration, the problems that Kubernetes solves, and some of the related high-level terminology. We’ll also see how storage for a Kubernetes cluster can be dynamically allocated using Cloud Volumes ONTAP and Trident, NetApp’s dynamic persistent volume provisioner for Kubernetes.
Read on below as we cover:
- What Are Containers? Some Background
- What Is Kubernetes?
- Containerization and Kubernetes
- Common Kubernetes Terminology
- Dynamic Storage Provisioning with Cloud Volumes ONTAP and NetApp Trident
This is part of an extensive series of guides about microservices.
What Are Containers? Some Background
On deployment, a container provides process and file system separation in much the same way as a virtual machine, but with considerable improvements in server efficiency. That efficiency allows a much greater density of containers to be co-located on the same host.
While container technology has been a part of Unix-like operating systems since the turn of the century, it was only with the advent of Docker that containers really came into the mainstream.
Docker has succeeded by bringing both standardization to container runtimes, for example, through the Open Container Initiative, and by creating a complete container management system around the raw technology, simplifying the process of creating and deploying containers for end users. Docker, however, can only be used to execute a container on a single host machine. That’s where Kubernetes steps in.
What Is Kubernetes?
Kubernetes makes it possible to execute multiple instances of a container across a number of machines and achieve fault tolerance and horizontal scale-out at the same time. Kubernetes was created by Google after over a decade of using container orchestration internally to operate their public services. Google had been using containers for a long time and developed their own proprietary solutions for data center container deployment and scaling. Kubernetes builds on those solutions as an open source project, enabling the world-wide community of software developers to grow the platform.
And Kubernetes introduction and implementation are growing. A recent bi-annual survey of over 2000 IT professionals from North America and Europe by the Cloud Native Computing Foundation found that 75% of respondents were using containers in production today, with the remaining number planning to use them in the future. Kubernetes usage has remained very strong with 83%, up from 77%, using the platform and 58% using it in production.
Containerization and Kubernetes
The ability to manage applications independently of infrastructure holds great value for cloud deployments. We can build out a cluster of machines in the cloud that provides the compute and storage resources for all of our applications, and then let Kubernetes ensure we get the best resource utilization. Kubernetes can also be configured to automatically scale the cluster up and down in response to changes in demand.
For deployed applications, Kubernetes offers many benefits, such as service discovery, load balancing, rolling updates, and much more. Kubernetes acts as an application server that is used to run all of the services, message queues, batch processes, database systems, caching services, etc. that make up an enterprise application deployment.
The flexibility of this service has driven Kubernetes introduction and adaptation across the cloud, with all major cloud vendors offering a native Kubernetes service, for example Amazon EKS, Google Kubernetes Engine, and Azure Kubernetes Service. Kubernetes is also the foundation of other container orchestration platforms, such as Red Hat OpenShift.
Common Kubernetes Terminology
There is a lot to understand when it comes to Kubernetes. In this section, we’ll give you an introduction to Kubernetes terminology that describes the main moving parts that make up the service.
- Cluster: The collective set of compute nodes and storage resources that form a Kubernetes environment. Each cluster has at least one control plane, which is responsible for overall management of the cluster, and a number of nodes on which containers will be scheduled to execute. Each node must have a container runtime installed, which is usually Docker, but may be an alternative, such as rkt.
- Pod: In the Kubernetes architecture, a set of containers may be deployed and scaled together. This is achieved by using pods, which are the minimum unit of deployment in a Kubernetes cluster, and allow more than one container to share the same resources, such as IP address, file systems, etc.
- Deployment: A deployment is used to control Kubernetes pod creation, updates, and scaling within the cluster, and is normally used for stateless applications. A stateless application does not depend on maintaining its own client session information, allowing any instance of the application to be equally capable of serving client requests.
- Stateful Sets: For certain types of applications, such as database systems, it is crucial to maintain the relationship between pods and data storage volumes. Stateful sets provide an alternative model to Kubernetes Deployments, and give each pod a unique and durable identity.
- Stateless Applications: Stateless applications do not keep a private record of client session information, which allows any running instance of the same application to process incoming requests. Applications deployed to Kubernetes or to containers in general are typically stateless, and so are easier to scale out horizontally across the cluster.
- Services: When multiple, interchangeable pod replicas are active at the same time, clients need a simple way to find any active pod they can send requests to. Services solve this problem by acting as gateway to a set of pods, which may even exist in different Kubernetes cluster.
Kubernetes Storage Terms
The following terms relate to storage provisioning in a Kubernetes cluster:
- Volume: A storage provisioned directly to a pod. Kubernetes supports a wide variety of volume types, including Amazon EBS, Azure Disk Storage, Google Persistent Disk, NFS, and many more. Volumes enable the containers within a pod to share information and are destroyed when their parent pod is deleted.
- Persistent Volume: A volume that exists independently of any specific pod and with its own lifetime. Persistent volumes can be used to support stateful applications, such as database services, enabling all components of an enterprise solution to be deployed and managed by Kubernetes. Another major advantage of using persistent volumes is that they insulate the developers creating pods from the lower-level implementation details of the storage they are accessing. This allows a persistent volume to be provisioned using a local disk, for example, on a developer workstation, and using Cloud Volumes ONTAP in production.
- Persistent Volume Claim: A claim for a persistent volume, which acts as the link between a pod and a persistent volume. A persistent volume claim is used by Kubernetes to search for suitable persistent volumes that can fulfil the request. This search is based on the size of the required storage, the access mode, a selector definition used to match against labels on the persistent volume, and, optionally, a storage class name.
- Storage Class: Storage classes add a further level of abstraction to storage provisioning by allowing persistent volume claims to only specify the type of storage they require. For example, slow, fast, and shared may all be valid user-defined storage classes. Storage class also encapsulates the details of the provisioner to be used, the type of volume to create, as well as other provisioner-specific settings. This gives much greater control to the cluster over how the storage is provisioned, which is why storage classes are generally used with dynamic storage provisioning.
- Dynamic Storage Provisioning: When Kubernetes administrators are required to manually set up persistent volumes ahead of time, this is known as static provisioning. By contrast, dynamic provisioning is used to automatically allocate persistent volumes based on the persistent volume claims that are received by the cluster. The type of storage to allocate is determined by the storage class specified in the claim.
- Provisioner: When using dynamic storage provisioning, each storage class will stipulate the provisioner that should be used to create new persistent volumes. Internal provisioners are provided by Kubernetes for a wide range of storage options, such as Amazon EBS, however, it’s also possible to specify external provisioners, such as NetApp Trident.
Dynamic Storage Provisioning with Cloud Volumes ONTAP and NetApp Trident
Cloud Volumes ONTAP brings the power and versatility of NetApp’s ONTAP storage systems to the cloud, using AWS, Google Cloud, or Azure compute and storage resources to create a virtual storage appliance. This provides a level of storage management that goes far beyond anything else currently available, featuring:
- Data Protection: Space efficient snapshots that can be created and restored in seconds for storage volumes of any size, providing enterprise data protection for applications running in Kubernetes.
- Storage Cloning: NetApp FlexClone® technology is used instantly provision writable, zero-capacity cost clones of existing storage volumes.
- Space Efficiency: Data deduplication, data Compression and thin provisioning are just some of the core components of Cloud Volumes ONTAP that reduce your cloud storage footprint and optimize operational costs for persistent volume storage.
- High Availability: Cloud Volumes ONTAP HA provides high availability for your data across Availability Zones with a RPO (Recovery Point Objective) of zero and a RTO (Recovery Time Objective) of less than 60 seconds.
- Data Replication: SnapMirror® provides highly efficient, block-level data replication between on-premises NetApp storage systems and Cloud Volumes ONTAP, which speeds up and simplifies the build out of cloud-based DR platforms. SnapMirror also works between instances of Cloud Volumes ONTAP for data availability across regions.
NetApp Trident connects the capabilities of Cloud Volumes ONTAP to Kubernetes by acting as a dynamic storage provisioner that allocates and makes available cloud storage in response to persistent volume claims. This alleviates the need for administrators to manually deploy and manage storage, as well as providing a very sophisticated feature set.
For example, using NetApp Trident, a persistent volume claim can be used to dynamically clone an existing persistent volume, as is typically required for DevOps CI/CD pipelines, as opposed to allocating a new storage volume. When the pod using the clone is destroyed, the cloned storage is also cleaned up automatically.
NetApp Trident and Cloud Volumes ONTAP can also be used to manage the storage for stateful sets, ensuring that pods are always bound to the same storage volumes. This is critical for deploying stateful applications, such as MySQL, to Kubernetes.
Conclusion
Kubernetes is today’s most widely-used platform for container and microservices orchestration and provides the scalability and flexibility required for deploying enterprise applications and services. Managing storage in a Kubernetes cluster with dynamic storage provisioning massively reduces the manual administration required for allocating cloud storage to pods and containers.
Using NetApp Trident, Kubernetes storage requests are dynamically fulfilled by Cloud Volumes ONTAP, which similarly does for storage what Kubernetes does for containers. Cloud Volumes ONTAP can also be used to allocate volumes for Docker containers directly using the nDVP plugin.
Learn more about how Cloud Volumes ONTAP supports Kubernetes Persistent Volume provisioning and management requirements of containerized workloads, and how Cloud Volumes ONTAP helps to address the challenges of containerized applications in these Kubernetes Workloads with Cloud Volumes ONTAP Case Studies.
See Additional Guides on Key Microservices Topics
Together with our content partners, we have authored in-depth guides on several other topics that can also be useful as you explore the world of microservices.
Application Mapping
Authored by CodeSee
- Bring visibility to your codebase
- What Is Software Mapping, 4 Types of Dependencies and How to Manage Them
- Why Code Dependencies Matter—and How to Identify and Resolve Them
Kubernetes in Azure
Authored by NetApp
- Azure Container Instance (ACI): The Basics and a Quick Tutorial
- Azure Container Instances vs AKS: How to Choose
- GKE vs AKS: Compared on Availability, Security, Tooling & More
Kubernetes on AWS
Authored by NetApp
- Kubernetes on AWS: 3 Container Orchestration Options
- AWS ECS in Depth: Architecture and Deployment Options
- AWS Kubernetes Cluster: Quick Setup with EC2 and EKS
