As a massive organization with terabytes, or even petabytes, of data adopting DevOps in the cloud, you want to take advantage of flexible, durable, extensible data across environments with faster non-disruptive delivery, accurate execution, automation and rapid recovery.
In this article we’ll take a closer look at how NetApp’s Cloud Volumes ONTAP and its FlexClone® feature work as a DevOps solution to reduce your development time and increase agility and TTM.
With Cloud Volumes ONTAP’s FlexClone® feature, enterprises can leverage NetApp’s data cloning technology to seamlessly ghost production data into a transparent writable layer that can be duplicated across environments for exact copies for testing, analytics, simulations, or any other purpose.
This functionality doesn’t sacrifice space or performance thanks to NetApp’s Write Anywhere File Layout (WAFL) and Snapshot® technology. When your deployment pipe is finished with automated testing and you are ready to push to production, FlexClone copies can be deleted and then refreshed on the next build process.
FlexClone capabilities are clutch in a number of cloud DevOps use cases, which we’ll look at here: hybrid cloud architectures, database analysis and development, and continuous integration/continuous delivery.
Traditionally, dev, QA, and production environments were separate and often disparate sets of data. Maintaining multiple subsets of “test data” in each environment meant trying to snapshot LUNs, move volumes and refresh data (all at increased cost).
For companies that move their data to cloud storage, there might be a challenge if they have 5, or 50, or 200 TB of data to mock and test for a release. Using a cloud offering to replicate data for testing is possible, but that means double the costs.
Why does this matter for cloud DevOps? The ability to clone data rapidly and provision writable clones of data that can be separated using FlexClone means your build pipelines don’t have to take shortcuts by testing on incomplete data subsets, your budget doesn’t get crushed paying for redundant data copies, and, though you have a single source of truth, your data is essentially stateless.
Statelessness is an important design paradigm in DevOps: it’s a term used for systems
How many times have you seen non-production environments being used as test data or test databases? How often are deployments tested successfully in staging only to fail in production? These things happen all the time. In these cases, data is typically the only difference.
At this stage, why make that sacrifice? Normally, copying a 10 TB Oracle database into a non-production copy meant using tools like GoldenGate, or refreshing databases with datapump, or something worse. It was costly, it was risky because of sync complications and it almost became a second “production” to manage.
Using Cloud Volumes ONTAP for cloud DevOps, it is much easier to create a FlexClone per environment from a production volume and mount it across environments. FlexClone creates a cloned volume, which is a writable virtual layer on top of a snapshot.
In terms of consuming storage, cloning takes only pointers and metadata copy. It's created instantly, and it consumes very little data, giving you a full virtual copy. It’s writable, but you’re paying only for the deltas of the data that you are writing. If one environment becomes corrupted, all you need to do is delete the clone, create another one, and your environment is back online. Or even better—have your build pipeline create new clones EVERY time you are pushing through a release.
The very nature of DevOps benefits is to take a complex challenge like this, put it in code, make it stateless, repeatable, and as lean as possible. Using Cloud Volumes ONTAP to govern your data between accounts or regions, provision your production resources, and FlexClone those volumes to your build pipeline environments is how you make your data durable and deployable.
In some environments, cloning is used to run hundreds of automated test scenarios, where each test needs its own clean copy. In these cases, cloning becomes an enabler for dev/test agility and an increased speed of TTM. You can see how DevOps benefits AWS here.
An inevitable reality in DevOps is the rollback. Whether it’s blue/green deployments, immutable, or a less flexible strategy, it happens. In most cases, data or database rollbacks are a recovery.
Using snapshotting and FlexClone allows you to quickly recover and throwaway changes by reverting back to previous versions. There are huge DevOps benefits in this because, in essence, the entire stack is rolled back in a more painless manner rather than bringing servers offline and restoring from Cloud Volumes ONTAP snapshots.
In a hybrid cloud architecture, a DevOps solution should be identical across both the on-premises and cloud environments. This includes your solution for data management.
Many companies, however, can struggle with the split and use different solutions per DevOps architecture. This leads to complexity of management (and teams), and inconsistent delivery processes and recovery procedures. Cloud Volumes ONTAP solves those data management problems with single-pane-of-glass manageability.
Whether you’re managing petabytes of on-premises NetApp arrays, public clouds infrastructure, or Cloud Volumes ONTAP instances, the solutions work from the same control panel, with the same features, and communicate to manage your data in the same ways through the use of the OnCommand Cloud Manager.
Using NetApp storage systems, this means you can move database and data volumes from your data center to the cloud without sacrificing speed, avoiding having to go to the trouble of building your own replication script (think rsync or robocopy), all while creating other native efficiencies.
Lastly, if your DR environment is in the cloud, it is a common burden to replicate data. Using Cloud Volumes ONTAP and FlexClone, your DR data can be cloned and tested as part of your DevOps build pipeline in your DR site, or DR region.
Let’s return to the 10 TB Oracle database in production. If you have only three
Building the same environment with NetApp FlexClone technology means that the database is fractionally over 10 TB and can be refreshed after deployments to keep environments consistent.
Using a production clone for OLAP is great, but what about the other huge DevOps benefits to testing database development changes on production data? What if you have an underperforming index on a table with millions or tens of millions of objects?
In that case you may want to build a new index, but you wouldn’t know how long that would take, or its impact.
Instead, you could simulate the load on a FlexClone copy of the database and build the index there to get a realistic impact on your performance before, during, and after, at which point you can delete the clone.
If you take backups of a database before making changes, or at the very least, backups of limited dumps of data, FlexClone can also come in handy. You can use FlexClone copies of the data, make the changes, test, then update pointers to the new database cluster, treating your data layer like an immutable deployment.
As these clones take very little time to create and have almost zero capacity, they save you the hours or even days it would take to create a copy. In our example above, that means saving 10 TB in capacity and costs.
A big shift in using NetApp for cloud DevOps benefits is extending your statelessness to the storage layer. This means making storage a large part of your deployment pipelines and leveraging a powerful and extensible solution to manage all of your environments: that solution is Cloud Volumes ONTAP and FlexClones. It’s about making your data changes less fearful, less wasteful, more automated, and more stateless. And it’s about shipping to production top to bottom with confidence.
One NetApp customer, Wirestorm, turned to Cloud Volumes ONTAP and FlexClone to create DevOps systems that are fast, efficient, and more unified. Cutting their provisioning time from 20 hours to just minutes has made it easier for them to deploy applications and develop the solutions their customers need. One Wirestorm DevOps manager hails the NetApp capabilities built into the platform, saying Cloud Volumes ONTAP “enables us to holistically deliver a solution that’s not piecemealing different tools from different vendors.”