Storage
Mobility
Protection
Analysis and control
TOPICShare
Amazon and Azure cloud compute environments require low-level access to storage at a block-level. These block-level devices are mounted by Amazon EC2 and Azure Virtual Machines just as a physical disk would be mounted by a bare-metal server, allowing the host to format and use the storage in any way required. In cloud environments, the connection between the compute node and the block-level device is typically an iSCSI connection using the iSCSI protocol.
Enterprise systems with many compute nodes may require hundreds or thousands of individual iSCSI storage allocations, with complex requirements, such as creating a single snapshot over a group of disks or performing application-consistent snapshots for database systems, for example. Other requirements may include standing up test environments with temporary copies of production data or managing pools of storage with different performance criteria.
In this article, we will examine the flexibility with which iSCSI storage can be managed with Cloud Volumes ONTAP and the benefits, such as storage space efficiencies, that come from doing so.
Using the iSCSI Protocol with Cloud Volumes ONTAP
There are three types of storage that Cloud Volumes ONTAP works with: CIFS, NFS, and iSCSI.
What is iSCSI?
iSCSI storage differs from CIFS and NFS shares in that the client host is able to manipulate individual blocks of data, instead of working primarily with files and folders. This is necessary for applications such as database systems in order to provide the required levels of performance and stability for the specific types of I/O operations they perform.
iSCSI devices can also act as boot disks for compute nodes, or simply as an attached physical disk, and can be formatted to any filesystem type, such as NTFS, ext4, etc.
Now that iSCSI has been explained, how does it work with Cloud Volumes ONTAP?
Cloud Volumes ONTAP is a virtual storage system that utilizes the compute and disk resources provided by the cloud platform on which it resides, whether that is AWS storage or Azure storage. In AWS, users can create storage pools, called aggregates, of varying performance using any of the available Amazon EBS disk types. In Azure, these aggregates can be created using either Premium Storage or Standard Storage. Tiering storage in this way makes it easier to manage the different types of disk allocations made across the enterprise and also the associated cloud storage costs.
Within an aggregate, volumes represent an individual allocation of storage. A volume can contain one or more LUNs, which are the iSCSI devices used by client host systems. Cloud Volumes ONTAP allows you to organize the iSCSI LUNs within a volume into QTrees, which can be thought of as a kind of directory. These mechanisms give you fine-grain control over the placement and organization of your iSCSI devices.
Snapshots with an iSCSI Configuration
A snapshot of all LUNs within a volume can be created at the same time with a single volume snapshot. This creates an instantaneous backup of all your iSCSI devices regardless of their number or size. Using Cloud Volumes ONTAP SnapRestore®, these snapshot backups can also be instantaneously restored. For more granular control, snapshots can be created of individual LUNs within a volume, with the ability to also perform restores at this level.
When working with block-level devices, there is very often a requirement to perform application-consistent snapshots. This is due to the caching of data on client systems and the potential for in-flight I/O operations to be only partially captured by the snapshot, if the application making use of the iSCSI device is not made aware that a snapshot is being performed. Snapshots created in this way are known as crash-consistent and may be subject to data corruption.
NetApp SnapCenter® integrates with Cloud Volumes ONTAP and a number of enterprise applications such as Microsoft SQL Server, Oracle, and Microsoft Exchange, to name a few, enabling you to easily create application-consistent snapshots of your iSCSI devices. SnapCenter provides a GUI interface, as well as a RESTful API, RBAC (Role-Based Access Control) for managing permissions, and can also be extended with custom plugins for any specific applications you may be using.
Cloud Volumes ONTAP Benefits
Using Cloud Volumes ONTAP to manage your iSCSI storage devices benefits you in a number of ways:
1. Granular control over storage organization: As described above, Cloud Volumes ONTAP allows you to create storage pools of varying disk performance and then further organize your iSCSI devices using volumes and qtrees. Having central control over all of your storage allocations makes managing large enterprise environments much easier.
2. Sophisticated snapshot capability: NetApp snapshots are more space efficient than both snapshot utilities on AWS and Azure. Amazon EBS snapshots, which require a full, baseline copy of the data before incremental snapshots can be created, and Azure Managed Disk snapshots, which are always a full copy of the source data. Cloud Volumes ONTAP snapshots:
- Are always incremental and instantaneous, regardless of source data size
- Allow for a single snapshot to be created of multiple iSCSI devices in a single volume, as well as snapshots of individual iSCSI devices
- Can be instantaneously restored using SnapRestore
- Readily support application-consistent snapshots using NetApp SnapCenter
- Can be mirrored and/or archived using Cloud Volumes ONTAP SnapMirror® and SnapVault® across hybrid and multicloud environments
- Allow older snapshots to be moved to object storage on Amazon S3 or Azure Blob storage using Cloud Volumes ONTAP’s Storage Tiering capability, thereby lowering cloud storage costs.
3. Rapid provisioning of DevOps and test environments: ONTAP Cloud can help you quickly spin-up DevOps and system test environments by using FlexClone®. This can be invaluable when working with large production systems, such as databases, that need to be duplicated in a temporary working environment in order to execute a suite of tests for a system upgrade or a software release. FlexClone can use the snapshots of your iSCSI devices to instantaneously create highly space-efficient, writable copies of your production data, which can then be connected up to the compute nodes in your test environment. When the test is complete, the FlexClone volume can simply be deleted. A FlexClone does not make a full copy of your source data when it is created, and instead only tracks that changes that you make to the clone. It is also possible to have multiple, active FlexClones for the same source volume at the same time.
4. Storage space efficiencies: Cloud Volumes ONTAP provides a range of built-in space efficiency technologies that can dramatically reduce your cloud storage footprint and costs. Data deduplication collapses multiple copies of the same block into a single block, which can be very useful for iSCSI devices that hold similar data, such as boot devices. Compression can be used to further reduce the amount of space used and can be very effective for database files. Thin provisioning can be used to present a larger device to a client host system than has actually been allocated, which can be useful in ensuring that space is not wasted by requests for large amounts of storage that do not end up being used. These technologies are all applied transparently without any changes needed on client systems and help you make the most effective use of your storage assets. Plus, tiered storage allows you to save by tiering cold, DR, and snapshot data to object storage when it isn’t in use and automatically bring it back to performant, higher-cost block storage only when it’s needed.
Conclusion
As we have seen in this article, Cloud Volumes ONTAP provides a feature-rich and enterprise-ready platform for provisioning iSCSI storage, bringing to the cloud the wealth of NetApp’s expertise in SAN technology and iSCSI performance for iSCSI SAN storage. This comes with the additional capabilities of CIFS and NFS file sharing, storage replication and archiving, high availability and failover in an AWS environment, data tiering to object storage, and more.
