Storage
Mobility
Protection
Analysis and control
TOPICShare
Current software development best practices emphasize frequent integration testing in order to catch problems as early as possible. This ensures that the source control branch used for the next release is always in a working state, and will not need be overhauled right before going to production. DevOps workflows implement CI (Continuous Integration) pipelines to perform end-to-end testing of applications and services on demand, which is typically triggered by a source code check-in. This testing requires a fully-functional environment in which to run, complete with compute, networking, and storage. Supplementing the environment with a test copy of production data, however, can be a tricky problem to solve.
In this article, we will demonstrate how Cloud Volumes ONTAP dramatically simplifies the provisioning of data storage clones for cloud-based testing, and how this can be automated using the most common tools and platforms currently used for DevOps.
DevOps Testing Environments with Cloud Volumes ONTAP
Automating the setup of software testing environments is one of the primary concerns of DevOps engineers. Not only are these environments important for CI and CD (Continuous Delivery & Continuous Deployment), which goes a step further than CI and ensures that applications are readily deployable to production, but also for software developers to perform independent testing of the application changes they are working on. An essential part of setting up a test environment is having access to test copies of production data, such as databases, as this will provide the most realistic conditions under which to test a new software release.
DevOps teams use a variety of tools for automating the setup of new environments, which allow them to treat Infrastructure as Code. Tools such as Ansible, Puppet, and Chef enable server and infrastructure deployments to be defined within a set of files, which can be used to recreate entire environments consistently, repeatably, and without human intervention. These files can also be managed and versioned under source control, just like application code. Ideally, during the setup of an environment for AWS automated testing, or a test environment in Azure, we would be able to easily create a cloud testing copy of production data alongside other infrastructure, and clean it up automatically at the end of a test run.
Integration with Cloud Volumes ONTAP
Cloud Volumes ONTAP is NetApp’s solution for managing enterprise data in AWS and Azure, and provides an extensive feature set that covers storage efficiency, scalability, and data protection, to name a few. By catering for NFS, SMB, and iSCSI block-level devices, Cloud Volumes ONTAP gives you unified control over data storage, in hybrid and multicloud environments.
Cloud Manager is the central platform for deploying and managing instances of Cloud Volumes ONTAP across an entire organization. Setting up a new instance of Cloud Volumes ONTAP can be completed in minutes using an innovative, web-based user interface. Discovering existing deployments, such as on-premises ONTAP systems, is trivial, and this allows for NetApp SnapMirror® replication to the cloud to be set up with a simple drag-and-drop operation.
The Cloud Manager working environments dashboard for a specific tenant.
Performing these operations through the UI makes it fast and easy to get started with Cloud Volumes ONTAP, however, advanced users need the additional ability to automate these operations as part of a wider orchestration plan. To facilitate this, Cloud Manager provides a REST API interface with full coverage of all available functionality. This gives you an integration point for external tools and processes over the ubiquitous HTTP protocol.
Data Cloning for Cloud Testing with Ansible and Cloud Volumes ONTAP
The feature of Cloud Volumes ONTAP that DevOps teams find most appealing is the ability to instantly create a writable, zero-capacity cost clone of any existing on-prem, AWS, or Azure storage volume, regardless of its size. This is achieved through NetApp FlexClone®, and can be used to create cloud testing copies of production databases, file shares, or any other type of storage residing in Cloud Volumes ONTAP. The clone only requires storage space for the data changes that are made to it, so initially requires no storage at all. After completing a test run, you can clean up the clone by simply deleting it. This saves a huge amount of developer time and capacity that can be put to better use. Saving time and saving money are the real benefits that come with using FlexClone.
Using the Cloud Volumes ONTAP REST API, we can integrate data cloning with the rest of the infrastructure setup for a test environment. Since REST has become the standard for client/server communication, this is possible from nearly every platform in the DevOps tool suite. For example, take a look at the following Ansible playbook that connects to Cloud Manager and automates cloning operations for cloud-based testing:
- hosts: 127.0.0.1
tasks:
- name: Login to Cloud Manager
uri:
url: http://<cloud manager server>/occm/api/auth/login
method: POST
body: '{ "email": "john@example.com","password": "secret" }'
body_format: json
status_code: 204
register: login
- name: Create a volume clone
Uri:
# production_data is the name of the volume to clone
url: http://<cloud manager server>/occm/api/vsa/volumes/<working environment>/<svm>/production_data/clone
method: POST
body: '{ "newVolumeName": "production_data_clone" }'
body_format: json
status_code: 204
headers:
Cookie: ""
# This would typically be done in a different playbook after all testing is complete
- name: Delete the volume clone
uri:
url: http://<cloud manager server>/occm/api/vsa/volumes/<working environment>/<svm>/production_data_clone
method: DELETE
body_format: json
status_code: 202
headers:
Cookie: ""
- name: Logout of Cloud Manager
uri:
url: http://<cloud manager server>/occm/api/auth/logout
method: POST
body_format: json
status_code: 204
headers:
Cookie: ""
These operations can be integrated with any other Ansible tasks, such as creating servers and deploying applications. This makes provisioning up-to-date test copies of production data a natural part of CI/CD pipelines and test environment management.
Conclusion
Cloud Volumes ONTAP provides an elegant solution for integrating the setup of test data with DevOps processes for cloud-based testing. Using NetApp FlexClone, multiple clones of the same storage volume can be made available for concurrent use, and with Cloud Manager’s REST API these clones can be created from tools such as Ansible and Chef. In fact, the deployment of Cloud Volumes ONTAP itself can be automated and integrated with Infrastructure as Code platforms, such as Terraform.
Find out more about cloud automation & testing software with data cloning and Cloud Volumes ONTAP, or start a free trial today in AWS or Azure.
