Storage
Mobility
Protection
Analysis and control
TOPICShare
In cloud-scale application development and deployment, DevOps and the CI/CD workflows integrated into DevOps pipelines play critical roles. By allowing code developed by different teams to be integrated and deployed in an incremental manner, CI/CD pipelines deliver the true value of agile.
However, with cloud scale growth comes a number of challenges, especially related to CI/CD data layer management. This blog will deep dive into the CI/CD use case and how Cloud Volumes ONTAP can help address CI/CD challenges.
In this blog we’ll cover:
- CI/CD Data Management Challenges
- How Cloud Volumes ONTAP Addresses CI/CD challenges
- Cloud Volumes ONTAP Benefits to CI/CD Pipelines
- How to Integrate Cloud Volumes ONTAP with Your CI/CD Pipelines
CI/CD Data Management Challenges
The speed at which features can be delivered to the customer is dependent on the effectiveness of the DevOps tool set and CI/CD workflows. There are different tools that can help you achieve the CI/CD efficiency you need in the cloud—the Openshift container platform, Bitbucket, Jenkins, Gitlab, GitHub, etc. However, when organizations focus on delivering outcomes through CI/CD, data management challenges often crop up as speed breakers.
Some of these challenges include:
- Development efficiency: Development process efficiency and reduced time to market (TTM) are critical indicators of success for any CI/CD workflow. There could be multiple simultaneous infrastructure-as-code integrations and deployments. The underlying data layer that supports CI/CD workflow should be well tuned to meet this demand.
- Heterogeneous environments: Cloud-scale deployments involve data management across all multiple environments, including dev/test, QA, production, and more. In the case of modern cloud deployments, the data could be spread across different cloud platforms or hybrid deployments that take advantage of existing on-premises architectures. The CI/CD process could require environments to be created on-demand to run locally or in the cloud to cater to such heterogeneous deployments.
- Storage costs: Unless managed efficiently, the environments required to run CI/CD workflows could become the biggest consumer of data storage in your operations. This in turn increases the associated storage costs that you’ll need to budget.
- Data availability: The stability and data availability of the environments created for developers to run the code could make or break the process lifecycle. The data layer must be highly available so that the continuous integration and deployment processes run uninterruptedly.
- Data protection: In the event of any data corruption, the environments should be restored back to a functional state as soon as possible and without any loss of data so that developer productivity isn’t affected.
- Repeatable and unified process: As cloud environments are agile, so should the CI/CD environments be catering to those deployments. Implementing repeatable and unified processes for managing the data layer for CI/CD processes becomes key in enabling this agility.
How Cloud Volumes ONTAP Addresses CI/CD Challenges
Cloud Volumes ONTAP is a data management solution that brings the capabilities of NetApp’s ONTAP software to the cloud of your choice. When integrated with your CI/CD processes, Cloud Volumes ONTAP helps address the common data management challenges organizations face when achieving agility in the cloud. It augments the storage layer through NetApp proprietary capabilities, thereby delivering resiliency, data protection and development efficiency irrespective of where your environments are located.
Cloud Volumes ONTAP Benefits to CI/CD Pipelines
Let’s look at some of the benefits offered by this approach:
- Reusability: Cloud Volumes ONTAP offers proprietary storage efficient FlexClone® technology that can be used to create space-efficient writable clones of preconfigured working environments and connect them to your self-hosted runners. This helps improve developer productivity and reduces TTM.
- Cost management: Cloud Volumes ONTAP provides native deduplication, thin provisioning, and compression features that bring down overall storage costs. You could spin up any number of environments as your application lifecycle demands, without burning a hole in your cloud budget.
- High availability: Cloud Volumes ONTAP is designed to be highly available, protecting from VM or cloud zone failures. Thus, access to CI/CD workflow environments used by self-hosted runners does not get interrupted, ensuring resilient application delivery.
- Data protection: Cloud Volumes ONTAP delivers data protection through NetApp Snapshot™ technology that creates point-in-time copies of data volumes. This can help protect from data corruption that could easily happen in highly volatile environments handled by CI/CD workflows. Using SnapRestore®, data can be recovered to a usable state to avoid disruptions
- Centralized management & data mobility: NetApp Cloud Manager can be used to manage the lifecycle of volumes across multiple environments, be in the cloud or on-premises. It can be used to replicate data between different environments using SnapMirror® data replication, thereby enabling customers to run self-hosted runners in an environment of their choice.
How to Integrate Cloud Volumes ONTAP with Your CI/CD Pipelines
All CI/CD workflows require a computational environment where the defined steps are executed when developers commit the code. Most DevOps tools provide managed shared environments which are non-customizable. However, to gain more flexibility, many customers opt for self-hosted runners that provide more control over the hosting environment. This approach provides end-to-end control of how the environment is configured, especially for the storage layer.
Cloud Volumes ONTAP integrates as the storage volumes of self-hosted runners deployed in the cloud of your choice or on-premises. For example, see the steps of how to set up your self-hosted GitHub runner in AWS. You need to spin up an AWS EC2 instance to install the same and then use it while executing GitHub Actions workflow. Integrating Cloud Volumes ONTAP into this architecture is as easy as creating an iSCSI volume through Cloud Manager and connecting it to your self-hosted runner EC2 instance.
To learn more about Github Actions for CI/CD, check out this blog on five Github CI/CD best practices, and read more about how enterprises are using Cloud Volumes ONTAP in these DevOps case studies and in these CI/CD customer case studies.
