Storage
Mobility
Protection
Analysis and control
TOPICShare
Subscribe to our blog
Thanks for subscribing to the blog.
NetApp’s new Cloud Tiering service is erasing the line between data centers and the cloud. By automatically moving cold data from All-Flash FAS (AFF) and hybrid FAS systems with SSD drives on-prem to object storage services in the cloud, Cloud Tiering makes it possible to devote storage systems to high performance workloads and extend CAPEX investments. But how does it work?
In this blog we will take a look at the internal architecture of the Cloud Tiering service so you can see exactly how we get infrequently used data off of your performant storage systems and into the cloud where it can be safely stored until it’s needed again.
The Cloud Tiering Service Components
The Cloud Tiering service architecture.
The Cloud Tiering service is built from a number of different components: an on-prem storage, a Cloud Tiering instance, a service connector, an object store in the cloud, and a secure HTTP connection to communicate between all of them. Let’s take a look at each of these in depth below.
1. The AFF or SSD-backed FAS System
First and foremost, there is the NetApp AFF or SSD-backed FAS storage in your on-prem data center where all of your data resides initially. These highly performant systems are at the forefront of enterprise application workloads and delivery due to their high speed processing, high-availability and durability and rich set of features. However, those benefits aren’t exactly necessary for storing cold data. That’s where the next component comes in.
2. The Cloud Tiering Instance
The Cloud Tiering service, leverages NetApp’s FabricPool data tiering technology and turns it into a managed service. The service UI is available through NetApp’s Cloud Central portal for all models of AFF or SSD-backed FAS, whether entry-level or high-end. To successfully configure Cloud Tiering, your storage system must have access to the internet and your version of ONTAP must be at least version 9.4. However, we recommend using the latest version, ONTAP 9.6, since it includes the new All volume tiering policy and many other new features which improve Cloud Tiering and ONTAP performance.
Powered by NetApp’s FabricPool technology, Cloud Tiering detects cold data on the on-prem system and then moves that data to an object storage service in the cloud, where it will stay until it becomes hot. Cloud Tiering communicates with your on-prem systems through a service connector, the next component we’ll discuss.
3. The Service Connector
The service connector is a software installed on a virtual machine within your VPC or Vnet or in your on-prem data center. The service connector allows the Cloud Tiering service to discover the on-prem ONTAP clusters and gather the relevant information needed. It serves as a proxy which relays communication through API commands securely over an HTTPS connection to the ONTAP cluster on one side, the object storage service (AWS S3, Azure Blob, and Google Cloud Storage) on the other side, and then back to the Cloud Tiering service.
4. Cloud Provider Object Storage
For now, three cloud storage providers can be used to tier inactive data: AWS, Azure and Google Cloud Storage. Cloud Tiering shifts cold data to object storage, which on AWS is available on Amazon S3 with Azure Blob on Azure and Google Cloud Storage on Google Cloud Platform.
On AWS, the Amazon S3 Standard or Standard-Infrequent Access storage classes can be used to accept tiered data. S3 Standard storage is designed for low latency, high throughput performance, 11 9’s durability of objects across multiple Availability Zones and 99.99 availability over a given year. S3 Standard also supports SSL for data transfer during transport and encryption of data at rest. Amazon S3 Standard-Infrequent access has almost the same features as S3 Standard but is because of a combination of low cost, and high performance is the best for long-term storage.
When Azure Blob storage is used as target storage for Cloud Tiering Hot access is selected by default. As well, Cloud Tiering supports tiering cold data to the Cool access tier.
With Google Cloud Storage, Cloud Tiering currently supports the Standard access tier. Nearline and Coldline access tiers are soon to follow.
5. Secure HTTP Connection
The final component to the Cloud Tiering architecture is the secure HTTP connection that will be used to carry out the data transfer between the data center and the cloud. TLS 1.2 is used for encryption, by the on-prem AFF or SSD-backed FAS systems directly to the object storage selected, when placing a new object containing cold data at the cloud tier and when cold data is re-warmed back to the on-prem performance tier.
How The Components Work Together
To start using the Cloud Tiering service, you’ll need to have an Amazon, Azure or Google account with access to the object storage service. Your next move is to subscribe to the Cloud Tiering service via the Cloud Tiering page on NetApp Cloud Central portal and create a service connector.
No configuration needed to be done on the storage controller side except if the networking requirements have not been met. Cloud Tiering automates all the necessary tasks that otherwise was DIY. Once the service connector was installed the ONTAP clusters can be discovered and added to Cloud Tiering UI and the object storage buckets can be created and attached to the ONTAP clusters. When all sides are configured, and the connection between the storage controller and object storage is established, Cloud Tiering allows to select the volumes, that contain data that should be tiered, and select the right tiering policy. From that point on, Cloud Tiering will begin transferring cold data to the cloud and back as specified by the tiering policy you selected.
You’ve Seen How it Works, Now Try It Out
With Cloud Tiering, you can use the cloud to maximize your data center capabilities. The different cloud tiering policies let you choose the type of data that will be tiered to the cloud. The Snapshot-only tiering policy transfers NetApp cold Snapshot data, the Auto-tiering policy lets ONTAP locate and transfer all the cold data it finds to cloud object storage and with the All tiering policy, all the data in a volume is considered cold and is shifted to object storage.
Whichever tiering policy you use, Cloud Tiering service helps free up on-prem NetApp storage space and get the most out of your equipment investment. As the demands for disk space rise every day, it can be hard to predict how much storage space to buy upfront and how to best predict future data usage. With AFF and SSD-backed FAS systems, those choices are easier to make since you know that Cloud Tiering makes it possible to offload inactive data to the cloud—in many cases this can be as much as 80% of the total storage footprint.
