Storage
Mobility
Protection
Analysis and control
TOPICShare
Ask any storage administrator or DevOps engineer—the quest to make their job easier is always one they’d undertake in a heartbeat. To frame this quest from my perspective, I’ll need to take you back 15 years, to my start as a systems and storage administrator at a university. Backing up our business-critical applications and research data was of the utmost importance, but nevertheless a menial task. Being a fully open-source shop at the time, our tools were those that were available to any systems administrator running Linux throughout the infrastructure: bash, cron, and rsync.
Our standard workflow looked something like this:
- Our scripts kick off an rsync to copy files at a scheduled time every night, capturing new data generated during the previous day.
- We scheduled weekly outage windows to stop all I/O and run an rsync update to ensure that we had an application- or data-consistent copy. Often this required shutting down a database for a few hours or disabling access to our NFS server to make sure that nothing changed during the final data sync.
- We had to maintain a remote storage system that was almost three times the size of the front-end system, to keep a month’s worth of weekly backups, and then back up the oldest copy to tape for longer archiving.
Looking back, the scripts got the job done, held together by chewing gum, a few open-source tools, and a team of crack sysadmins. Although I have some fond memories of those days, looking back, none of it was built for maintainability or upgrades, and we ran into several issues that led us to seek an enterprise storage solution:
- The amount of data to be backed up increased exponentially, and weekly scheduled maintenance windows for data sync became untenable.
- Researchers and developers did not appreciate the scheduled backup windows because they disrupted the workflow for an extended period of time.
- The amount of time required to restore data from tape in the event of a data loss incident became unacceptable.
All of these challenges led us to start looking for our own easy button.
We addressed these issues with our first NetApp® storage system, a FAS3070. NetApp Snapshot™ technology meant that local backups took up practically no space, and NetApp SnapMirror® allowed data to be rapidly transferred to a remote ONTAP® system and synchronized regularly in an automated fashion. Application consistency was maintained at the time through SnapManager® plug-ins for both Oracle databases and VMware vSphere virtual machines, which quiesced the systems before making a near-instantaneous Snapshot copy. This approach eliminated the need for scheduled shutdown of services for any considerable period. Having NetApp and its suite of tools in our environment truly gave us the easy button we were looking for.
Now, to tell you another chapter in the story…
Today, just over a decade after I first became a NetApp customer, I work for the company as a hybrid cloud solutions architect. I test, validate, and develop solutions that use NetApp’s suite of tools in the open-source and container arena.
Recently we published a new solution, Red Hat OpenShift with NetApp. We have been working with Red Hat’s enterprise Kubernetes platform to test and validate the integration of NetApp storage systems and data management tools for Kubernetes. These tools include the NetApp Astra™ Trident storage orchestrator for Kubernetes and NetApp Astra Control Center, our user-friendly complete application management tool for Kubernetes.
In May 2021, before the release of Astra Control Center for Red Hat OpenShift, we published a video demo of a manual application migration process. The migration was from one OpenShift cluster with a NetApp ONTAP storage back end to another remote site with similar infrastructure. This video demonstrates a containerized WordPress deployment that uses NetApp Astra Trident to provision persistent volumes, which store data for the webpage front end and the MariaDB database. The persistent volumes are replicated to the secondary ONTAP system by using NetApp SnapMirror® technology and imported as persistent volumes into the remote deployment of Red Hat OpenShift by using the volume import functionality built into Astra Trident. The WordPress application is then redeployed referencing the imported persistent volumes, and our application successfully stands up on our remote cluster.
The demonstrated workflow allows us to essentially clone an application from one site to another simply and reliably by using NetApp tools for data management. However, it still requires multiple manual steps and is not non-disruptive to our application’s running state:
- We must stop the application on the primary OpenShift cluster and make a Snapshot copy of the persistent volume data on the primary ONTAP system to get an application-consistent copy.
- We must use SnapMirror to mirror the data volume to the secondary ONTAP cluster, then break the mirror relationship so that the volume becomes read/write.
- We must run a NetApp Astra Trident import operation to convert the replicated ONTAP data volume to a persistent volume that is accessible by Kubernetes.
- We must redeploy the application on the secondary cluster by using a modified manifest to import the replicated persistent volumes.
- To simplify the visualization of the workflow, we must license the Advanced Cluster Management for the Kubernetes add-on to the OpenShift Container platform so that we can see both deployments simultaneously.
Although this process accomplishes the goal of cloning our application from one site to another rapidly and efficiently, it is still a potentially disruptive manual process with multiple steps, and inherently not easy to accomplish.
So the question remains, how do we make this easy? Can we quiesce our app and clone it to a remote cluster without disruption? Can we accomplish this without having to manually mirror, import, and redeploy our application? Just as NetApp data management technology came along years ago and made my former job much easier, NetApp has done it again for containerized applications with NetApp Astra Control Center.
As part of the Red Hat OpenShift with NetApp solution, we have a new demo that replicates the workflow from our original demo, again using a WordPress deployment with a website front end and a MariaDB back end. However, this time we use NetApp Astra Control Center to show how it makes things so much easier.
If we compare this demo with the manual steps in the first demo, we see quite a dramatic difference:
- Astra Control Center can act as a single pane of glass to visualize both OpenShift clusters, storage back ends for both ONTAP systems, and all applications deployed at both sites, capturing their running state when it performs a backup.
- There is no requirement for a SnapMirror or Advanced Management for Kubernetes license, or for the configuration that goes along with it.
- Instead of a manual process that requires edits to YAML and multiple CLI commands, we can protect our application and clone it from one cluster to another remote cluster with just a few clicks.
Not only is the workflow easier, there is practically no disruption to our application. The application at the initial site is briefly quiesced but remains running for the entire backup process. The application is backed up locally, pivoted through an object store, and automatically redeployed on the remote OpenShift cluster in a running state, all with a single click.
Comparing the two workflows side by side shows the efficiency that Astra Control Center brings to the flow.
|
Steps |
Without Astra Control manual migrate |
With Astra Control clone |
|
1 |
Quiesce or stop the application. |
Run backup and clone operation. |
|
2 |
Make a Snapshot copy of the persistent volume. |
|
|
3 |
Use SnapMirror to mirror the Snapshot copy to the remote site. |
|
|
4 |
Break the mirror or use NetApp FlexClone® to make the copy writeable. |
|
|
5 |
Use Trident to import to a new PV at the remote site. |
|
|
6 |
Redeploy the application using the imported PV. |
|
You want to talk about easy… that’s easy.
To find out more about NetApp Astra Control Center, or to try out your own demo, go to /astra.