Storage
Mobility
Protection
Analysis and control
TOPICShare
Cloud Volumes ONTAP deployments in AWS make use of AWS EBS volumes as the underlying cloud storage layer for delivery of its enterprise-class data management services. These volumes come in a variety of types which you can choose between based on your requirements for performance and capacity.
Now, with the advent of storage tiering for Cloud Volumes ONTAP, NetApp extends this flexibility by enabling you to use Amazon S3 as a capacity storage tier for your “cold” data.
Storage tiering for Cloud Volumes ONTAP works in by using Amazon EBS as your performance tier for your “hot” data in conjunction with Amazon S3 as the capacity tier for your “cold” data.
Cloud Volumes ONTAP automatically moves data between these tiers as required, which dramatically reduces costs for cold, infrequently-accessed data, while still providing rapid access to hot data. Having the ability to store the large bulk of your data in lower-cost Amazon S3 storage, until it’s actually needed, is ideal for setting up cloud-based DR environments.
In this article we’ll look at how Cloud Volumes ONTAP storage tiering works and how to set it up. We’ll also, as an example, examine the DR use case for the technology showing the costs benefits of data tiering compared to using Amazon EBS for DR storage.
What Is Tiered Storage?
Tiered storage is a way for data to be kept in the storage format that is the most effective for its data usage pattern, at any given time. While this concept exists in the on-prem data center, in the cloud it is much easier, faster, and effective to shift data between optimally priced object storage and performant block storage repositories.
Capacity Tiering with Amazon S3
Starting with ONTAP 9.2, storage volumes can be created on flash disk aggregates that are backed by tiered object storage architecture with the addition of the FabricPool feature. For Cloud Volumes ONTAP, these aggregates can use either General Purpose SSD (gp2) or Throughput Optimized HDD (st1) EBS volumes as the performance tier, with Amazon S3 as the capacity tier.
When a volume is created on such as aggregate, Cloud Volumes ONTAP will automatically move data that it considers cold over to Amazon S3 and free up the associated Amazon EBS storage. If the data stored on the Capacity Tier (Amazon S3) is accessed, it is transparently pulled back to the Performance Tier (Amazon EBS). Each volume therefore has part of its storage allocation in Amazon EBS and the other part in Amazon S3.
Setting up Cloud Volumes ONTAP storage tiering is very simple. The only requirement is to create a VPC endpoint to the Amazon S3 service. This ensures secure, encrypted communication to and from your Amazon S3 buckets over the AWS network. Cloud Volumes ONTAP automatically takes care of the rest, including the allocation of an Amazon S3 bucket for your data. There’s no further manual configuration or tweaking required—it just works.
Cloud Disaster Recovery Use Case
When setting up a DR environment, storage is required for a lot of data that would normally be accessed only in an emergency, either when there has been some kind of disaster or catastrophic system failure.
In the ideal case, we’d be able to store this data on lower-cost storage while it is not being actively used, and move it back to high performance storage when it actually needs to be accessed. This would reduce the upfront cost of setting up our cloud-based DR environment by deferring the cost of allocating high performance Amazon EBS storage until it’s actually needed. Using the new Cloud Volumes ONTAP storage tiering feature, we can do exactly that.
SnapMirror is NetApp’s solution for efficient and incremental storage replication, which can be used to synchronize data from on-premises NetApp appliances to Cloud Volumes ONTAP, such as would be required for a cloud-based DR environment.
When a SnapMirror destination volume is stored on a tiered aggregate, Cloud Volumes ONTAP will automatically push the majority of the data to Amazon S3, retaining only metadata information about the volume in Amazon EBS. This remains the case for new data that arrives incrementally.
If the SnapMirror relationship is broken and the destination volume are accessed, Cloud Volumes ONTAP will transparently pull only the data that are required from Amazon S3 back into Amazon EBS. If the SnapMirror relationship is subsequently re-established, the data that was pulled onto the Amazon EBS performance tier will be transparently moved back to the Amazon S3 capacity tier, freeing up the space used in the performance tier.
This data movement to and from the Amazon EBS and S3 tiers is managed automatically by Cloud Volumes ONTAP without any user intervention required.
Cost Comparison
One of the main drivers for using Cloud Volumes ONTAP storage tiering is the reduction in cloud storage costs for infrequently-accessed data.
Let’s now consider a simple example in which 10TB of such storage is required. The monthly storage cost comparison is given in the table below.
|
Amazon EBS (gp2) |
Amazon EBS (st1) |
Amazon S3 Standard |
|
|
Capacity |
10TB |
10TB |
10TB |
|
Pricing |
$0.10 per GB-month |
$0.045 per GB-month |
$0.023 per GB |
|
Cost / Month |
$1024 |
$460.80 |
$235.52 |
The AWS region used for this pricing was US East (N. Virginia), with the assumption that the storage would be allocated for an entire month. In this AWS region, the price for S3 Standard is lower when more storage is allocated, falling to $0.022 per GB for storage over 50TB and $0.021 for allocated storage over 500TB.
The costs above relate purely to data storage, as there are also access request costs related to Amazon S3. However, as the Amazon S3 tier will be storing cold data, less access to the data would normally be expected.
As we can see, there is a huge price differential between using Amazon EBS and Amazon S3. Storage costs for Amazon S3 Standard are a quarter of the cost of General Purpose SSD (gp2). The cost savings for Throughput Optimized HDD (st1) are also significant, with Amazon S3 costs being roughly half the price. This makes for a very tangible reduction in cloud storage costs for an Cloud Volumes ONTAP environment.
Summary
NetApp has further enhanced data management in the cloud by offering the ability to seamlessly tier cold data to and from less expensive Amazon S3 capacity storage. In this article, we’ve shown how easy this is to set up and use and the huge savings in cloud storage costs that result, especially Cloud Volumes ONTAP DR environments.
As well as data replicated using SnapMirror, Cloud Volumes ONTAP storage tiering is also able to move infrequently-accessed data from ONTAP Snapshots over to Amazon S3. This, again, happens automatically when the snapshot data is deemed to be cold. In the future, support will also extend to the tiering of cold data in active filesystems.
