Storage
Mobility
Protection
Analysis and control
TOPICShare
TL;DR Kubernetes applications can be protected with Astra Control directly from a CI/CD pipeline. Skip to a brief intro to Astra Control, a pipeline demo and Jenkins sample pipeline by clicking here.
Over the past several years, organizations that build software have been embracing the many concepts of DevOps. DevOps practices break down organizational barriers, bringing development and operations teams closer together. DevOps practices also empower the teams to accelerate feature delivery, increase availability, and make services and applications more stable, improving the team’s productivity. Automation is one key ingredient of success— from building, testing, and operating applications at scale to running a fully automated infrastructure platform or stack.
Container technology, and Kubernetes as the leading container orchestrator, not only powers these new applications and services, but also enables teams to achieve many of the DevOps goals and automated workflows. Kubernetes has become the DevOps platform. A common reason to use automated workflows is to enable continuous integration and continuous deployment (CI/CD) pipelines that build, integrate, and run automated test suites on applications as developers commit new code. DevOps engineers and site reliability engineers (SREs) will likely have a few pipelines that cater to the various workflows for new feature development, regression testing, bug fixes, quality engineering, and other functions in the development process. The teams are distilling an automated process from manual and semi-automated operations.
As teams increase their level of automation, the pace of change to in-production applications can feel overwhelming. Therefore, some teams prefer to protect the in-production applications or services. They don’t only want to protect the code and container images; they also want to protect application state and configuration data (such as Kubernetes objects and resources associated with the application) as well as an application’s persistent data.
In this blog post, we’ll take a closer look at a promotion-to-production pipeline that deploys a new version of an application—first into a staging environment, and then into a production environment. This example applies equally to the major public clouds or on-premises environments. Although we’re showing a deployment of one version of the app, the pipeline can also be used with other strategies, such as blue/green or canary deployment.
As part of the CI/CD pipeline, we’re going to protect the application by creating a complete application backup. An application-aware backup of the in-production application and its data, state, and configuration can be useful for numerous DevOps workflows. However, the purpose of this example is to allow the application or service team to roll back the change to the production environment and restore service quickly if something breaks.
At the end of this blog post, you’ll find a video that shows the whole process in action. You’ll also find a link to the resources to replicate the process in your environment.
Building a CI/CD pipeline that protects real-world in-production apps
The application that’s being built and deployed to a Kubernetes cluster is based on Magento, an e-commerce solution with a web-based front end; an Elasticsearch instance for search and analysis features; and a MariaDB database that tracks all the shopping inventory and transaction details. Every pod in the application uses persistent volumes to store data. The persistent volumes are automatically created by NetApp® Astra™ Trident, the Container Storage Interface–compliant storage orchestrator for Kubernetes that enables storage to be provisioned on Azure NetApp Files, NetApp Cloud Volumes Service for Google Cloud, and on-premises NetApp ONTAP® storage systems. To use the data protection capabilities, the pipeline integrates with Astra Control and Astra Trident, but more on that later. We’re using Jenkins, a popular tool for CI/CD pipelines, to automate the build, test, and deployment of the application.
A version of the application that was deployed by using the appropriate chart from the Bitnami Helm chart repository is running in both staging and production environments. We begin with our currently deployed version of our app.
Our promotion-to-production pipeline starts by downloading the application code from its repository, builds a container image from the Dockerfile, and finally pushes that image to our image registry. When these stages are complete, the pipeline continues to gather the prerequisites to perform the update of the application in the staging and production environments and the data protection tasks. These prerequisites include Helm, to upgrade the chart of the application in case there are major changes between versions, and the NetApp Astra toolkit, to create the application-aware backup in the staging and production environment.
In backing up the staging and production environments, the pipeline stages use NetApp Astra Control to create backups before new code is deployed. The backups consist of application-consistent copies of the persistent data along with the active state and configuration of the Kubernetes resources used in the application.
After the application is backed up in the staging and protection environments, the new build is deployed into the staging environment. Upon completion of that deployment, the pipeline stops to ask for active approval before promoting the new build to production.
You can verify that the new version of the application is running in the production environment by checking the Magento configuration pages after the pipeline has completed.
Application-aware backups with Astra Control
Because the entire application has been backed up as part of the promotion-to-production pipeline, teams can feel more confident about highly automated application deployments. This approach can be easily applied to other DevOps tooling; it isn’t exclusive to Jenkins.
NetApp Astra enables the data management capabilities for applications in Kubernetes. Astra Trident integrates NetApp storage into the Kubernetes clusters, and Astra Control enables application and application-aware data management across the Kubernetes cluster in the environment.
To take advantage of these management capabilities, you must first register your Kubernetes clusters with Astra Control so that it can complete an inventory. Subsequently, applications are automatically discovered whenever possible (by app label; custom definitions are possible, of course). After you select the applications to be managed, the data management capabilities become available in the UI and can also be automated. Backups of applications are stored in predefined Amazon S3 compatible storage buckets by Astra Control.
In addition to the automation as part of the pipeline, Astra Control provides a web-based UI and a REST API. Using these interfaces, application, storage, and backup administrators can see the application environments across multiple Kubernetes clusters, including the storage back ends. To automate the data management capabilities, the REST API or the Python SDK in the Astra toolkits can be used.
As later stages of the pipeline run, the application view in Astra Control will show the backups that were created: status (healthy) and available actions for the backup (restore backup, delete backup). In addition, you can see the container images and versions used in the application.
Astra Control enables users to create application specific steps before and/or after snapshots, backups and restores using scripts called execution hooks. For example a hook can be used to ensure a database is quiesced before a snapshot and unquiesced afterwards to ensure application consistency.
NetApp created the open source 'Verda' project to provide curated scripts for several commonly used applications such as mariadb, these scripts are a great starting point for customers to customize the scripts for their own needs. To learn more please visit https://github.com/NetApp/Verda.
Try it yourself to see it in action
If you want to see the pipeline in action, watch the video below.
If you’d rather try it yourself, you can find the examples in the NetApp Astra toolkits on GitHub (alternatively: git clone https://github.com/NetApp/netapp-astra-toolkits.git).
For more information about NetApp Astra Control, including how to use it in your development projects and try it free, visit our Astra page. And be sure to check the NetApp Cloud blog for future DevOps workflows with NetApp Astra. To learn more about the underlying infrastructure in the environment, visit the Red Hat OpenShift with NetApp solutions page.