Storage
Mobility
Protection
Analysis and control
TOPICShare
Kubernetes remains the standard-bearer in container orchestration. But there are other options available out there. Though not as popular as Kubernetes, HashiCorp’s Nomad is quickly catching up and is now becoming a serious disruptor in how containers are being orchestrated. How does this development affect your Kubernetes storage layer?
In this article we’ll take a side-by-side look at Kubernetes vs. Nomad, delving into the various features, benefits, use cases, and challenges of both.
Jump down using the links below to read where we cover:
The Basics of Container Orchestration
Containers are ephemeral by nature: they exist to run a single process, and then they are terminated. These are also built to be immutable, meaning that application upgrades require those containers to be replaced, rather than updated. Since they are virtual software packages intended to run on any platform, containers don’t have native access to computing resources such as storage or networking.
Container orchestration tools were developed to provide frameworks for managing microservices at scale through container lifecycle management procedures. These platforms eliminate the manual processes involved in ensuring containers are functional in production, such as:
- Host configuration and scheduling
- Resource allocation
- Maintaining high availability
- Scaling containers based on workload demands
- Traffic routing and load balancing
- Security practices such as authentication and access control
In the modern cloud-native landscape, Kubernetes and Nomad are two popular container orchestration tools. Now let’s have a closer look at how these two similar services compare.
Kubernetes
Kubernetes is an extensible, portable, and highly efficient open-source container orchestration platform that enables the management of containers across multiple hosts. The platform helps organizations achieve large-scale infrastructure management through declarative deployment and automation.
On account of the list of features and benefits to schedule and deploy containerized applications on different clusters of physical or virtual machines, Kubernetes has quickly become the de facto standard container orchestration platform for organizations leveraging the modern DevOps and CI/CD frameworks.
Some Kubernetes features include:
- Automated pod scheduling
- Self-healing properties
- Rollbacks and automatic rollouts
- Loosely-coupled infrastructure
- Application-centric management
- Platform-agnostic deployments
Kubernetes Architecture
At a high level, the Kubernetes architecture can be likened to a layered system with the top layers abstracting complexities of the underlying layers. This architecture consists of components that work together to deliver various features and functions we described above.
A typical Kubernetes Cluster Architecture (Source: Kubernetes.io)
The key components of the Kubernetes cluster architecture include:
- Control plane
- Compute machines (master and worker nodes)
- Pods
Nomad
HashiCorp’s Nomad is a simple, flexible workload orchestration tool that facilitates the deployment, management, and scaling of different workload types across multiple regions and cloud platforms. Nomad provides a common pool of infrastructure from multiple platforms—on premises and in the cloud—to be shared between various applications.
Instead of being attributed as a specialized container orchestration and automation platform, Nomad focuses on container scheduling and cluster management through optimized resource utilization with bin-packing.
Some features of Nomad include:
- Mixed Application Deployment
- Single, self-contained binary agent
- Native support for multi-region federation
- Device plugins and GPU support
- Designed with the Unix philosophy and integrates seamlessly with the HashiCorp ecosystem
Nomad Architecture
Nomad clusters leverage Consul for configuring and discovering cluster services. A Nomad cluster is composed of various servers (between three and seven) connecting with client agents through remote procedure calls. The cluster infrastructure is divided into regions that manage one or more availability zones or data centers, where regions are loosely coupled, and communicate with each other using Gossip Protocol Architecture.
Nomad Reference Architecture (Source: HashiCorp)
Kubernetes vs. Nomad: Side-by-Side Comparison
Kubernetes is an end-to-end container orchestration platform that relies on a dynamic ecosystem of various loosely-coupled components. Nomad is architecturally much simpler, though offers the same features a robust orchestrator offers. The two platforms share a number of features and differ in several ways. Let us take a closer look.
Kubernetes vs Nomad: Similarities
Kubernetes and Nomad are both open-source tools that are built for container orchestration and to support similar use cases. As a result, both carry a number of common features:
Automated Scheduling
By leveraging a preset-function-based ranking system, both Nomad and Kubernetes automatically lookup suitable hosts to run an application.
- Kubernetes uses filters and scores to keep a ranking of feasible machines that pods can bind to. The platform uses factors such as resource requirements, inter-workload interference, policy constraints, and affinity specifications to keep a record of healthy nodes.
- Nomad automates workload scheduling by using an evaluation process to change a node’s state and match it with a job’s desired state.
Auto-Recovery and Self-Healing Deployments
Both orchestration platforms include various tools and mechanisms to restart applications and recover data in case of a failure.
- Kubernetes reschedules pods to the healthiest available node in case one worker machine fails. Kubernetes deployment environments also include capabilities like scale sets and autoscaling groups that enable self-healing nodes.
- Nomad ensures self-healing by checking every node’s emergent state. A failure in the client node’s state triggers a new evaluation, after which Nomad tries to reconcile the emergent state with the system’s desired state.
Rollout and Rollback Strategies
To manage large cluster deployments, both Nomad and Kubernetes support rolling updates out of the box.
- Kubernetes uses ReplicaSets to maintain an application's desired state. With rolling updates as one of its default deployment strategies, Kubernetes allows a gradual process to replace pods, one at a time. Through rollbacks, Kubernetes also allows the deployment to be updated to an earlier desired state.
- Nomad enables rolling updates by limiting parallelism and interfacing with HashiCorp’s monitoring to determine the health of the service. In cases of deployment failures or unhealthy configurations, Nomad automatically rolls back to an older healthy service based on the defined update strategy.
Nomad and Kubernetes Storage Orchestration
Both Nomad and Kubernetes support various third-party plugins that conform to the Container Storage Interface (CSI) standard, allowing applications to attach with external storage volumes. By allowing storage consumption from public/private clouds or bare-metal on-premises infrastructure, both platforms allow efficient storage orchestration to maintain stateful workloads.
Kubernetes vs Nomad: Differences
Though both platforms are meant for container orchestration, Nomad and Kubernetes have a few fundamental differences, including:
Flexible Workload Support
Kubernetes is built to orchestrate containers hosting workloads, while Nomad is considered more of a workload scheduler that can be used to manage the deployment of both legacy and containerized applications, and scheduling batch jobs.
End-to-end Container Orchestration
Kubernetes offers end-to-end container orchestration services including load balancing, configuration management, routing, feature gates, and service discovery. Nomad is primarily a task scheduling platform that can orchestrate different workload types as an extended feature.
Scalability
Both platforms are built for autoscaling in large deployments, however Nomad is considered favorably to scale larger clusters than Kubernetes. While Kubernetes is built to support clusters with up to 5,000 nodes orchestrating a maximum of 300,000 containers, Nomad can scale clusters exceeding 10,000 nodes in production, and surpassed the benchmark for the two million container challenge.
Deployment Consistency
Kubernetes offers different deployment and management environments to make it easier for organizations to perform development and testing before pushing apps to deployment. These may create inconsistencies in configuration and capabilities when going live. Nomad uses a single, portable library that can be deployed on any environment to provide the same user experience across different platforms.
Kubernetes: Pros and Cons
Kubernetes Advantages
- Extensive community support
- Various distributions on all major cloud providers
- Enhanced portability and flexibility
- Multicloud support
- Open source
Kubernetes Disadvantages
- Highly complex ecosystem
- May incur runaway costs
- Requires the configuration of numerous interoperating components to manage and deploy
- Requires in-depth knowledge of new overlay networking models
- Built mainly to support Linux-based containers
Nomad: Pros and Cons
Nomad Advantages
- Simple to learn and use
- Requires less configuration management, and is also considered suitable for smaller teams
- Works with existing networking models
- Ability to schedule, deploy, and manage different workload types
- Being platform agnostic, supports both Windows- and Linux-based containers
Nomad Disadvantages
- Nomad is still in its infancy as compared to the maturity of Kubernetes
- Narrow tooling options
- Vendor lock-in: integrates deeply with the HashiCorp suite
- Lacks wider contribution and community support
Which Should You Use?
Both Nomad and Kubernetes offer different, yet excellent platforms for operationalizing containerized workloads. Due to their differences in form and function, there are certain use cases when one offers more than the other.
Nomad is a general purpose orchestrating platform that can be used to manage clusters of legacy applications, traditional batch jobs, or even supplement an existing Kubernetes ecosystem by supporting a multi-orchestrator framework. The platform also is designed to be platform-agnostic by supporting multiple environments, including Windows. Its simplicity also makes it ideal for organizations just getting started with container orchestration.
Kubernetes, on the other hand, is a mature framework with comprehensive container orchestration services and tools integrations that is developed with a primary focus to support Linux-based containers. The platform is considered perfect for organizations already running applications on Linux containers and have the right skillset to manage its complex ecosystem.
Summary
With the rising adoption of containers, the significance of container orchestrators in the modern cloud-native era continues to rise. With over 74% of containerized environments running on a Kubernetes distribution, the platform continues to be the clear winner among the list of container orchestrators. As compared with Kubernetes, Nomad isn’t a close competitor yet but is an emerging disruptor in the container orchestrator world, offering simpler, more scalable deployments using a single, portable binary agent.
Both platforms, while built to orchestrate containers in clusters, have different features and functions that support different use-cases. As with any other form of technology, it is difficult to predict what comes next, but it will be interesting to see how the two platforms fare in the years to come.
To further optimize your containerized Kubernetes storage, NetApp offers Cloud Volumes ONTAP, the data management platform for AWS, Google Cloud, and Azure. With Cloud Volumes ONTAP, Kubernetes users get persistent storage for containers with the benefits of
- Automatic persistent volume claim responses through the Astra Trident provisioner
- Data protection and high availability capabilities to prevent data loss and enable quick recovery
- Storage efficiencies that can reduce persistent volume storage costs by 70% or more
- Instant zero-capacity volume cloning with FlexClone® technology
For more details, see how Cloud Volumes ONTAP is being used in these companies’ Kubernetes deployments.
Q&As
● Is Nomad better than Kubernetes?
The answer to this depends on a number of factors and the required use case. While Nomad is platform-agnostic and more flexible, Kubernetes offers support for only specific platform types and has been inherently built to run Linux containers. While Kubernetes is specifically a cluster management platform, Nomad is known to be a more general-purpose tool rather than focusing on orchestration.
Due to the importance of container orchestrators in cloud-native clusters, it is highly recommended that the list of features, and the pros and cons of both platforms are appropriately weighed before choosing one of them.
● Why use Nomad instead of Kubernetes?
Nomad is a toolkit designed specifically for building highly scalable clusters. In addition to being easy to learn and understand, Nomad makes it possible to run stateful workloads without having to worry about managing complex configuration files.
Nomad also has certain features that may appeal to those who want to run database clusters. For instance, Nomad supports multi-master replication between nodes which makes it easier to recover after node failures. In addition, Nomad offers several advanced scheduling options including priority queues and weighted randomization.
● Is Nomad based on Kubernetes?
Nomad is not based on Kubernetes technology. Nomad is a stand-alone platform by HashiCorp that acts as an alternative to Kubernetes. Both Kubernetes and Nomad are built for container orchestration and offer various features that support specific use cases.
