hamburger icon close icon
Azure File Storage

Address Persistent Storage Woes in Azure Kubernetes Service

More about Azure File Storage

Subscribe to our blog

February 12, 2020

Topics: Azure NetApp Files Advanced7 minute read

Container-based workloads have become commonplace in the cloud, with enterprises showing a clear preference toward Kubernetes as an orchestration platform. Microsoft’s managed Kubernetes service, Azure Kubernetes Service (AKS), enables deployment, configuration, and management of the Kubernetes control plane through the Azure platform.

AKS offers users the flexibility to simply lift and shift their on-premises container-based workloads to the cloud and to focus on green-field deployments without having to invest time setting up and managing the Kubernetes service in Azure.

While pods are ephemeral and can be created and destroyed anytime, data access requires a stable underlying storage layer to meet performance, durability, and high availability workload demands. Azure NetApp Files (ANF) is a Microsoft first-party managed file service in Azure powered by NetApp® ONTAP technologies and sold and supported by Microsoft. In addition to databases, DevOps, HPC, and virtual desktop workloads, ANF can be used as persistent storage for workloads hosted in AKS.

In this blog, I discuss the challenges enterprises face with persistent storage in AKS and how ANF can help address them without compromising user experience or Kubernetes performance.

Persistent Storage for AKS and Challenges

The advantage of container-based microservices architecture is the flexibility offered by the service in terms of portability and idempotency of deployments. For example, the same pod definition can be used to deploy the services in an on-premises Kubernetes cluster as well as in AKS. This makes deployment and migration of microservice-based applications to Azure easy.

As containers are stateless, they depend on external persistent storage volumes for data management. Applications of varying complexity can therefore be deployed in AKS as the storage is being decoupled from the process. This process, however, comes with its challenges. The performance of applications deployed in containers is dependent on the backend service that provides Kubernetes persistent storage. Let’s explore some of the challenges organizations face when planning storage for AKS.

High Availability

High data availability is non-negotiable for all line-of-business applications. Because microservices data is stored in external persistent volumes in AKS, high availability must be managed at the storage service level, with no administrative overhead. Developers need not worry about designing for high data availability; rather, this should be inherently addressed by the storage features.

Efficiency

It is a common architecture for multiple pods to have shared access to volumes, and the storage service should provide this shared access. Customers should be able to provision shared persistent volumes of the required size using existing processes and without compromising quality or performance. The storage service should also be able to support on-demand scalability based on data growth.

Durability

The application’s data layers should be durable and resilient to any level of failure from an underlying cloud resource such as a hardware, network, or disk failure. This is especially important when using a microservices architecture for databases. Because the data resides in the persistent volumes, integrity of transactions is a must for ACID-compliant (atomicity, consistency, isolation, and durability) databases.

Data Backup

Having a proper backup mechanism for Kubernetes persistent volumes is crucial for ensuring protection from accidental data loss and corruption. Though pods can be created as needed using pod definition files, without a usable copy of the data application, recovery cannot be completed to meet the defined recovery point objective (RPO) and recovery time objective (RTO).

Performance

When monolithic applications give way to microservices, the user experience is dependent on multiple services and how well each service performs. Data operations, or IOPS, and throughput benchmarks vary for different microservices, and meeting these demands requires a flexible storage service. Because performance requirements can change throughout the application lifecycle, the attached volume should also be capable of meeting these varying performance throughput demands on the fly.

Azure Kubernetes Service Storage Configuration

Persistent storage plays a crucial role in microservices architecture in which data must be decoupled from the pod lifecycle. AKS supports multiple options for configuring persistent storage including Azure Disks, Azure Files, and Azure NetApp Files. These persistent storage options are differentiated by the “StorageClass” shipped with Kubernetes. The StorageClass concept is similar to the different storage profiles that can be made available to pods in an AKS cluster—each with their own set of properties, quality of services, and policies.

Azure Disks

Persistent volumes for pods in AKS can be created with Azure Disks, which use Azure premium or standard storage. Premium storage provides performance levels at the same level as SSDs, and standard storage provides HDD-level performance. While the former is ideal for production workloads with higher IOPS requirements, HDD storage is more suited for test and development environments. Persistent volumes created from Azure Disks can, however, be mounted to just one pod at a time and do not support use cases that require shared storage.

Azure Files

Azure Files is a managed file share service for creating SMB and NFS file shares accessible to workloads hosted on-premises or in Azure. The service can be used to create persistent volumes for pods deployed in AKS clusters. Like Azure disks, Azure Files also support SSD and HDD capabilities through premium and standard storage options. AKS cluster versions, however, must be higher than 1.13 in order to use premium storage. Storage volumes created from Azure Files can be accessed simultaneously from multiple pods in the cluster.

Azure NetApp Files

While Azure Files and Azure Disks are backed by Azure Storage, ANF is powered by NetApp® ONTAP’s trusted storage management technology. Available as a first-party service in Azure, ANF can be managed from the portal through Azure CLI, PowerShell, or REST API calls. Managed NFS and SMB file shares created with ANF can be mounted as persistent storage in AKS pods. Moreover, persistent volumes created through the service support shared storage access from more than one pod.

Containers: The Azure NetApp Files Advantage

NetApp’s proven expertise in enterprise storage management makes ANF the ideal NFS managed file share service in Azure. The service’s NFS storage class allows for easy integration with AKS, with no service-specific configuration or learning curve involved in the process. For example, on-premises container-based workloads that use NFS file shares for persistent volumes can continue with the same configurations in Azure or by simply updating the pod definition to use an ANF volume.

ANF offers built-in advanced features to address the major challenges around AKS persistent storage:

Availability: ANF’s out-of-the-box, built-in high availability means AKS administrators need not worry about designing and implementing storage-level high availability of persistent volumes.

Accessing persistent volumes: Multiple pods can access the same persistent volume created using ANF, thus meeting the shared storage requirements commonly associated with microservices based architecture. The feature is natively supported in ANF and is available as a plug-and-play model, offering unparalleled efficiency.

Scalability: ANF volumes can scale from 100 GB to a maximum capacity of 100 TB, meaning services deployed in pods are no longer impaired by limitations in scalability.

Snapshots: ANF persistent volumes can be protected using Snapshots, a point-in-time data backup that takes almost no new storage capacity, making them extremely efficient. In the event of data loss or during disaster recovery, the snapshots can be restored to create new ANF volumes that can then be attached back to new pods in the AKS cluster.

Durability: ANF ensures 99.9999999% data durability, protecting data stored in persistent volumes from any kind of hardware, network, or disk-related failures. This helps maintain the integrity of data transactions, which is crucial for database workloads.

Performance: With three performance tiers—Standard, Premium, and Ultra—you can right-size the storage performance to the pod requirements.  Even performance-intensive workloads like databases can be deployed in AKS pods with confidence. The Standard tier provides a throughput of 16MB/second, the Premium a throughput of 64 MB/second, and the Ultra tier 128 MB/second.

Boost Your Microservices Deployment

The integration of ANF persistent volumes in AKS deployments grants developers the flexibility they need to focus on application development without having to worry about underlying infrastructure settings and storage configuration. ANF’s enterprise-class persistent storage for microservices deployed in AKS offers unparalleled performance, availability, scalability, and durability.

Overcome the challenges of persistent storage in AKS without compromising user experience or performance. Subscribe to Azure NetApp Files today.
Jeff Whitaker

Cloud Data Services

-