More about Cloud Migration
- Azure StorSimple EOL: Using Cloud Volumes ONTAP for Hybrid Cloud Management
- Refactoring Applications to Kubernetes in Cloud Migrations
- The War Over the Cloud Has Ended (and the Winner Might Surprise You)
- Modern Data Estate: What IT Leaders Need To Know
- Strategies for AWS Migration: The New 7th R Explained
- Cloud Computing Deployment Models and Architectures
- Cloud Adoption Strategy: What’s the Best Approach for Your Organization?
- What is Cloud Migration? Strategy, Process and Tools
- Hybrid Cloud Strategy: A Winning Model for Enterprises
- Google Cloud Architecture and Building Your Own Solution Architecture
- SnapMirror in the Cloud: New Use Cases for NetApp’s Data Replication Technology
- 8 Digital Transformation Technologies and Their Business Impact
- What Is Digital Transformation in Banking?
- Digital Transformation in Healthcare: 4 Key Trends
- Digital Transformation: Examples from 5 Industries
- The Future of Cloud Computing: 5 Trends You Must Know About
- 5 Types of Digital Transformation and the Technologies that Power Them
- Digital Transformation Strategy: 6 Tips for Success
- Kubernetes Data Mobility with Cloud Volumes ONTAP
- Application Migration to Azure: 4 Approaches and One Migration Tool
- Cloud First Strategy: Challenges, Considerations, and Best Practices
- Application Migration to AWS: Free Tools to Ease Your Migration
- Why Cloud Adoption Fails and 6 Tips for Success
- Cloud Application Migration: A Practical Guide
- AWS Snowball vs Snowmobile: Data -Migration Options Compared
- AWS Snowball Edge: Data Shipping and Compute at the Edge
- AWS Snowmobile: Migrate Data to the Cloud With the World’s Biggest Hard Disk
- AWS Snowball Family: Options, Process, and Best Practices
- AWS Cloud Adoption Framework (CAF): 6 Migration Perspectives
- Top 3 Cloud Adoption Frameworks: Your Path To The Cloud
- AWS Snowball Pricing Simplified
- Azure Cloud Adoption Framework: The 9 Methodologies Explained
- Cloud Roadmap: Mapping Out Your Path To The Cloud
- Hybrid Deployment on Google Cloud: Meet Google Anthos
- 3 Ways to Create an Azure Migrate Project
- Azure Migration Step by Step: Discover, Migrate, Optimize, and Monitor
- Migrate from VMware to Azure: The Basics and a Quick Tutorial
- Migrate SQL Server to Azure: Options, Tools, and a Quick Tutorial
- Migrate Databases to Azure: 3 Quick Tutorials
- 4 Ways to Migrate SQL to Azure
- Azure Migration Program: 4 Key Elements
- Azure Migrate: Key Components and a 4-Step Migration Plan
- Cloud Journey: 6 Stages of Cloud Adoption
- 5 Azure Data Migration Tools You Should Be Using
- Azure Migration Tools: One-Click Migration for VMs and Data
- Typical Mistakes and Misconceptions Business Leaders Have About Hybrid and Multicloud
- Google Cloud PostgreSQL: Managed or Self-Managed?
- AWS Storage Gateway: Connecting Your On-Premise Storage to the Amazon Cloud
- Azure PostgreSQL: Managed or Self-Managed?
- Cloud Scalability: How Cloud Volumes ONTAP Stores Petabytes of Data
- AWS Migration: Understanding the Process and Solving 5 Key Challenges
- Google Cloud Pricing vs AWS: A Fair Comparison?
- VMware Cloud Services: A New Option for Hybrid Cloud Management
- AWS PostgreSQL: Managed or Self-Managed?
- Google Cloud MySQL: MySQL as a Service vs. Self Managed in the Cloud
- Azure MySQL: MySQL as a Service vs. Self-Managed in the Cloud
- AWS MySQL: MySQL as a Service vs. Self Managed in the Cloud
- Cloud Adoption for Financial Companies
- VMware on AWS: Architecture and Service Options
- VMware on Google Cloud: A Deployment Roadmap
- VMware on Azure: One-Step Migration to the Cloud
- Better in the Cloud: Workloads Gartner Says You Should Move to the Cloud Now
- 3 Cloud Migration Approaches and Their Pros and Cons
- Oracle on Google Cloud: Two Deployment Options
- How to Upload Files to Azure Blob Storage with AzCopy, PowerShell, and More
- Oracle on Azure: Fully Managed vs Self Managed
- Azure SQL Server: Managed Service vs Self Managed
- Google Partners with NetApp on New Bare Metal Solution in the Cloud
- AWS Database Migration Service: Copy-Paste Your Database to Amazon
- SQL Server on Google Cloud: Two Deployment Options
- GCP Migration with Cloud Volumes ONTAP
- SQL Server in AWS: Two Deployment Options
- Azure Database Migration Service: Automate Your Cloud DB Migration
- Azure Managed Service Provider: How to Save Time and Reduce Cloud Overhead
- Hybrid Cloud Storage: The Best of Both Worlds
- Hybrid Cloud Storage Solutions for AWS Compared: Storage Gateway vs Cloud Volumes ONTAP
- AWS Oracle: Two Options for Migrating Your Oracle DB to Amazon
- Migrating Physical Server to AWS: Now Free with AWS CloudEndure
- Azure Hybrid Cloud:
Azure in Your Local Data Center
- What Is a Lift and Shift Cloud Migration?
- Google Cloud Migration Tools: Copying 1GB or 500TB? Learn How
- VMC on AWS Vs. Cloud Volumes ONTAP
- AWS Cloud Migration Services: Don’t Migrate Alone
- 5 Steps to the Cloud: AWS Migration Checklist
- Setting up Storage Gateway with Amazon EC2
- VMware Cloud on AWS: How Fujitsu Saves Millions Using Cloud Volumes ONTAP
- SQL Server End of Life Challenges and How Cloud Volumes ONTAP Can Help
- The Cloud Tiering Service Architecture: How We Get Cold Data from Your Data Center to the Cloud
- Cloud Data Integration 101: Benefits, Challenges, and Tools
- Google Cloud Migration: Why and How to Make the Move
- What’s the Easiest Way to Start Using the Cloud? Three Cloud Onboarding Approaches
- AWS vs Azure vs Google Cloud: Choosing the Best Cloud Provider for You
- Cloud Migration Tools: Transferring Your Data with Ease
- Azure Migration Strategy: Four Steps to the Cloud
- 11-Step Azure Migration Checklist
- AWS Migration Strategy: The 6 Rs in Depth
- Officeworks Adopts a Cloud-First Strategy with Cloud Volumes ONTAP
- Azure Migration: The Keys to a Successful Enterprise Migration to Azure
- Cloud Volumes ONTAP: Cloud Migration Case Studies
- One Cloud Out of Many: Why Enterprises Are Turning to Multicloud and Hybrid Cloud Architectures
January 19, 2020
Topics: Cloud Tiering Data TieringAdvanced6 minute readCloud Volumes Service for GCP
NetApp’s new Cloud Tiering service is erasing the line between data centers and the cloud. By automatically moving cold data from All-Flash FAS (AFF) and hybrid FAS systems with SSD drives on-prem to object storage services in the cloud, Cloud Tiering makes it possible to devote storage systems to high performance workloads and extend CAPEX investments. But how does it work?
In this blog we will take a look at the internal architecture of the Cloud Tiering service so you can see exactly how we get infrequently used data off of your performant storage systems and into the cloud where it can be safely stored until it’s needed again.
The Cloud Tiering Service Components
The Cloud Tiering service architecture.
The Cloud Tiering service is built from a number of different components: an on-prem storage, a Cloud Tiering instance, a service connector, an object store in the cloud, and a secure HTTP connection to communicate between all of them. Let’s take a look at each of these in depth below.
1. The AFF or SSD-backed FAS System
First and foremost, there is the NetApp AFF or SSD-backed FAS storage in your on-prem data center where all of your data resides initially. These highly performant systems are at the forefront of enterprise application workloads and delivery due to their high speed processing, high-availability and durability and rich set of features. However, those benefits aren’t exactly necessary for storing cold data. That’s where the next component comes in.
2. The Cloud Tiering Instance
The Cloud Tiering service, leverages NetApp’s FabricPool data tiering technology and turns it into a managed service. The service UI is available through NetApp’s Cloud Central portal for all models of AFF or SSD-backed FAS, whether entry-level or high-end. To successfully configure Cloud Tiering, your storage system must have access to the internet and your version of ONTAP must be at least version 9.4. However, we recommend using the latest version, ONTAP 9.6, since it includes the new All volume tiering policy and many other new features which improve Cloud Tiering and ONTAP performance.
Powered by NetApp’s FabricPool technology, Cloud Tiering detects cold data on the on-prem system and then moves that data to an object storage service in the cloud, where it will stay until it becomes hot. Cloud Tiering communicates with your on-prem systems through a service connector, the next component we’ll discuss.
3. The Service Connector
The service connector is a software installed on a virtual machine within your VPC or Vnet or in your on-prem data center. The service connector allows the Cloud Tiering service to discover the on-prem ONTAP clusters and gather the relevant information needed. It serves as a proxy which relays communication through API commands securely over an HTTPS connection to the ONTAP cluster on one side, the object storage service (AWS S3, Azure Blob, and Google Cloud Storage) on the other side, and then back to the Cloud Tiering service.
4. Cloud Provider Object Storage
For now, three cloud storage providers can be used to tier inactive data: AWS, Azure and Google Cloud Storage. Cloud Tiering shifts cold data to object storage, which on AWS is available on Amazon S3 with Azure Blob on Azure and Google Cloud Storage on Google Cloud Platform.
On AWS, the Amazon S3 Standard or Standard-Infrequent Access storage classes can be used to accept tiered data. S3 Standard storage is designed for low latency, high throughput performance, 11 9’s durability of objects across multiple Availability Zones and 99.99 availability over a given year. S3 Standard also supports SSL for data transfer during transport and encryption of data at rest. Amazon S3 Standard-Infrequent access has almost the same features as S3 Standard but is because of a combination of low cost, and high performance is the best for long-term storage.
When Azure Blob storage is used as target storage for Cloud Tiering Hot access is selected by default. As well, Cloud Tiering supports tiering cold data to the Cool access tier.
With Google Cloud Storage, Cloud Tiering currently supports the Standard access tier. Nearline and Coldline access tiers are soon to follow.
5. Secure HTTP Connection
The final component to the Cloud Tiering architecture is the secure HTTP connection that will be used to carry out the data transfer between the data center and the cloud. TLS 1.2 is used for encryption, by the on-prem AFF or SSD-backed FAS systems directly to the object storage selected, when placing a new object containing cold data at the cloud tier and when cold data is re-warmed back to the on-prem performance tier.
How The Components Work Together
To start using the Cloud Tiering service, you’ll need to have an Amazon, Azure or Google account with access to the object storage service. Your next move is to subscribe to the Cloud Tiering service via the Cloud Tiering page on NetApp Cloud Central portal and create a service connector.
No configuration needed to be done on the storage controller side except if the networking requirements have not been met. Cloud Tiering automates all the necessary tasks that otherwise was DIY. Once the service connector was installed the ONTAP clusters can be discovered and added to Cloud Tiering UI and the object storage buckets can be created and attached to the ONTAP clusters. When all sides are configured, and the connection between the storage controller and object storage is established, Cloud Tiering allows to select the volumes, that contain data that should be tiered, and select the right tiering policy. From that point on, Cloud Tiering will begin transferring cold data to the cloud and back as specified by the tiering policy you selected.
You’ve Seen How it Works, Now Try It Out
With Cloud Tiering, you can use the cloud to maximize your data center capabilities. The different cloud tiering policies let you choose the type of data that will be tiered to the cloud. The Snapshot-only tiering policy transfers NetApp cold Snapshot data, the Auto-tiering policy lets ONTAP locate and transfer all the cold data it finds to cloud object storage and with the All tiering policy, all the data in a volume is considered cold and is shifted to object storage.
Whichever tiering policy you use, Cloud Tiering service helps free up on-prem NetApp storage space and get the most out of your equipment investment. As the demands for disk space rise every day, it can be hard to predict how much storage space to buy upfront and how to best predict future data usage. With AFF and SSD-backed FAS systems, those choices are easier to make since you know that Cloud Tiering makes it possible to offload inactive data to the cloud—in many cases this can be as much as 80% of the total storage footprint.