What Is Kubernetes Statefulsets? Statefulset vs. Deployment

September 22, 2021

Topics: Cloud Volumes ONTAP Elementary 8 minute read Kubernetes

What Is Kubernetes StatefulSet?

A StatefulSet is a Kubernetes API object for managing stateful application workloads. StatefulSets handle the deployment and scaling of sets of Kubernetes pods, providing guarantees about their uniqueness and ordering.

If you want to provide workload persistence using Kubernetes persistent storage, you can incorporate a StatefulSet into your solution. A pod in a StatefulSet can fail, but the persistent pod identifier will enable its replacement with a new pod that matches the existing storage volume.

What is the difference between StatefulSet and deployment?
Similar to deployments, StatefulSets manage pods with identical container specifications. They differ in terms of maintaining a persistent identity for each pod. While the pods are all created based on the same spec, they are not interchangeable, so each pod is given a persistent identifier that is maintained through rescheduling.

This is part of an extensive series of guides about microservices.

Related content: Read our guide to Kubernetes StatefulSet vs Deployment

In this article:

Running Stateful Applications on Kubernetes with StatefulSets
How Do Pods Work in a Kubernetes StatefulSet?
Creating StatefulSets
Kubernetes Storage with Cloud Volumes ONTAP

Running Stateful Applications on Kubernetes with StatefulSets

You can leverage the StatefulSet model for the deployment of a stateful application in Kubernetes. A StatefulSet serves as a deployment object specifically designed for stateful applications. As with other deployments or ReplicaSets, StatefulSets manage the deployment of pods created from a particular container spec. However, they differ from regular deployment, in that they allow you to specify the dependencies and order of the deployment.

Benefits of a StatefulSet deployment include:

Unique identifiers—every pod in the StatefulSet is assigned a unique, stable network identity, consisting of a hostname based on the application name and instance number. For example, a StatefulSet for a web application with three instances may have pods labeled web1, web2 and web3.
Persistent storage—every pod has its own stable, persistent volume, either by default or as defined per storage class. When the pods in a cluster are scaled down or deleted, their associated volumes are not lost, and the data persists. Unneeded resources can be purged by scaling down the StatefulSet to 0 before deleting the unused pods.
Ordered deployment and scaling—the pods in a StatefulSet are created and deployed in order, according to their increments. Pods are also shut down in (reverse) order, ensuring that the deployment and runtime are reliable and repeatable. The StatefulSet won’t scale until all every required pod is running, so if a pod fails, it will recreate the pod before it attempts to add more instances as per the scaling requirements.
Automated, ordered updates—a StatefulSets can handle rolling updates, shutting down each node and rebuilding it according to the original order, until every node has been replaced and the older versions cleaned up. The persistent volumes can be reused, so data is migrated to the new version automatically.

Limitations of Kubernetes StatefulSets include:

Support in Kubernetes versions—Kubernetes versions preceding 1.5 do not support StatefulSets, as this was a beta resource until version 1.9.
Storage provisioning—unless an admin pre-provisions storage, you need a persistent volume provisioner to provision storage for specific pods based on the desired storage class.
No automatic purge—for data safety purposes, associated volumes are not purged when a StatefulSet is scaled down or deleted.
Headless service—a StatefulSet requires a headless service to handle the network identity of its pods. You have to create this service.
No termination guarantees—you cannot expect pods to be terminated upon the deletion of a StatefulSet. To ensure pods are terminated in an ordered manner, you need to scale the set down to 0 before you delete it.

Related content: Read our guide to Kubernetes storage provisioning

How Do Pods Work in a Kubernetes StatefulSet?

Each pod in a StatefulSet has a unique identity comprising a stable network ID, an ordinal and stable storage. The pod retains this identity regardless of the node it is scheduled (or rescheduled) on.

Stable Network Identity

Each pod in a StatefulSet derives its hostnames from the name of the StatefulSet and the pod’s ordinal. The constructed hostname follows the $(statefulset name)-$(ordinal) pattern.

For example, if three pods are created in a StatefulSet for a web application, they might be named web-0,web-1 and web-2. The StatefulSet can control its pods’ domains using a headless service. The domains managed by the service take the form of $(service name).$(namespace).svc.cluster.local, with cluster.local as the domain of the cluster. When a pod is created, it is assigned a DNS subdomain to match, in the form of $(podname).$(governing service domain), with the serviceName defining the governing service.

Depending on how your cluster’s DNS is configured, it might not be possible to search the DNS name of new pods immediately. This can be the case when another client in the cluster has already sent a query for a pod’s hostname before the pod was created. The negative caching typical in DNS ensures that any result of a failed search is remembered and can be reused, even once the pod is running, for several seconds at least.

There are several options if you need to find pods immediately when they are created. You can directly query the Kubernetes API, instead of relying on DNS searches. You can also reduce the caching time in the Kubernetes DNS provider, which typically involves modifying the CoreDNS config map to cache for 30 seconds.

Ordinal Index

In StatefulSets with N replicas, every pod is given an integer ordinal between 0 and N-1, which is unique with its set.

Stable Storage

Kubernetes generates a PersistentVolume for every VolumeClaimTemplate. For example, you can configure Kubernetes to give each pod a persistent volume according to storage class, with specified amounts of storage being provisioned for each class. If there is no specified storage class, the default class will apply. When pods are scheduled (or rescheduled) onto nodes, their volumeMounts will mount the volumes associated with their PersistentVolume Claims. The volume associated with a claim is not deleted upon the deletion of a pod or StatefulSet, so you have to delete persistent volumes manually.

Pod-Name Label

When a new pod is created by the StatefulSet Controller, it is given a statefulset.kubernetes.io/pod-name label, which reflects the name of the pod. Labels enable you to attach services to specific pods in your StatefulSet.

Creating StatefulSets

You can use kubectl apply to create a StatefulSet.

Once the StatefulSet is created, it ensures that the required number of pods are running and available at any time. StatefulSets automatically replace any pod that fails or is ejected from its node and creates a new pod that is automatically associated with the original pod’s specification. This includes provisioning the same storage resources and applying various configurations such as resource requests and limits.

The following example describes a manifest file for a StatefulSet and a service. The example was shared by Google Cloud.

k8s-statefulset-netapp-1

To clarify:

This will create a service object called nginx, which is indicated by metadata: name. This service targets an application called nginx, which is indicated by labels: app: nginx and selector: app: nginx. The service exposes port 80, naming it web, and controls the network domain and routes network traffic to the application deployed by the StatefulSet.
A StatefulSet called web is created, containing three replica pods—replicas: 3.
The pod template, spec: template, specifies that pods are labelled as app: nginx.
The pod specification, template: spec, specifies that the pods in the StatefulSet run a single container, nginx, which runs the 0.8 version of the nginx-slim container image hosted in the container registry.
The pod specification indicates the use of the web port that the service opens.
A mount path is specified by template: spec: volumeMounts and is called www. This path indicates where storage volumes should be mounted within the container.
The StatefulSet creates a PersistentVolumeClaim, www, provisioning 1GB of storage.

In short, the pod specification involves these instructions:

Label every pod as app: nginx.
Run one container called nginx in each pod.
Run the 0.8 version of the nginx-slim
Direct pods to use port 80.
Save the data to the www mount path.

Kubernetes Storage with Cloud Volumes ONTAP

NetApp Cloud Volumes ONTAP, the leading enterprise-grade storage management solution, delivers secure, proven storage management services on AWS, Azure and Google Cloud. Cloud Volumes ONTAP capacity can scale into the petabytes, and it supports various use cases such as file services, databases, DevOps or any other enterprise workload, with a strong set of features including high availability, data protection, storage efficiencies, Kubernetes integration, and more.

In particular, Cloud Volumes ONTAP supports Managing Stateful Applications in Kubernetes and Kubernetes Persistent Volume provisioning and management requirements of containerized workloads.

Learn more about how Cloud Volumes ONTAP helps to address the challenges of containerized applications in these Kubernetes Workloads with Cloud Volumes ONTAP Case Studies.

See Additional Guides on Key Microservices Topics

Together with our content partners, we have authored in-depth guides on several other topics that can also be useful as you explore the world of microservices.

Kubernetes StatefulSet: A Practical Guide

Subscribe to our blog

September 22, 2021

Topics: Cloud Volumes ONTAP Elementary 8 minute read Kubernetes