The enterprise storage landscape caters to different types of data, with part of this data being actively used by applications on a day-to-day basis, and another part of it remaining unused or untouched for long periods of time. Data used by line-of-business (LOB) applications and databases belong to the former category while archive data, backups, etc. belong to the latter.
To ensure that you’re using your storage capacity optimally, you need to match your data with the storage that best fits its performance requirements, all while keeping the cost in check. In the Azure cloud, Azure Blob storage offers three different tiers catering to different types of data that will help you do that.
This blog will show you step by step how to switch between different Azure storage tiers based on the lifecycle of the data so you can make effective usage of your cloud storage investment, and the further optimization that you can get with NetApp® Cloud Volumes ONTAP.
Azure offers three storage tiers to store data in blob storage: Hot Access tier, Cool Access tier, and Archive tier. These tiers target data at different stages of its lifecycle and offer cost-effective storage options for different use cases.
Hot Access Tier: This tier should be used for the data frequently accessed by applications, and that is read or written very often. The data storage costs for this tier are higher than other tiers, but access charges are lower, thus suiting the target use case.
Cool Access Tier: This storage tier is suitable for data that is not accessed frequently by applications and is expected to be stored for a minimum of 30 days; for example, for short term storage of backup, telemetry data, media files, etc. The cost of using the cool access tier is lower than using the hot tier, however access costs and availability are not the same. This tier also has a lower SLA (99%) when compared to hot tier (99.99%), and there is a per-GB charge for accessing the data.
Archive Tier: As the name indicates, this tier is intended to be used to store rarely accessed or archival data. Its storage rate is the cheapest on Azure, but the data retrieval charges are the highest. This tier is intended as Azure archive storage for data that is expected to be stored for a minimum of 180 days and that can tolerate a larger retrieval window. The Archive tier is useful storage for large amounts of inactive data, such as data required to be stored for compliance purpose, long-term backup, archived datasets, etc.
These three data tiers are supported in General Purpose storage V2 and Blob storage accounts. Customers can also convert from existing GPv1 storage account to GPv2 from the Azure portal to get the benefits of storage tiering.
Now let’s take a look at how to switch between these three storage tiers so you can put your data in the tier that best matches its usage pattern.
Note: While the Hot and Cool tiers can be enabled at the storage account level or at the blob level, the Archive tier can only be enabled at the blob level. All three storage access tiers can exist in the same storage account and the default tier for a blob is inherited from the account level setting. However, you could also use blob-level tiering to set the tier at the object level.
1. First, let’s look at how the storage tier can be set when the storage account is initially created. The account level tier is selected when you create the storage account. In the Azure portal, select Create a resource->Storage->Storage account. Select from the options of Hot or Cool for the Access tier (default) setting.
Note: This tier will be inherited by any blob created in the storage account.
2. To switch the storage access tier at a later point, browse to the storage account-> configuration->Change Access tier(default) to Cool and click “Save.”
3. Once you save your settings, you can confirm by browsing to the configuration of the same storage account. Here you’ll be able to see that the tier is currently set to Cool. Similarly, you could switch to the Hot Access tier if the storage account was originally created in the Cool tier.
4. To switch the tier of a specific blob inside the storage account, browse to the Storage account->Blob service->Blobs-><container name>-> select the blob-> and click on the context menu on the left-hand side. From here, select “Blob properties.”
5. When the Blob Properties window opens up, click on “Change tier” on the top menu bar.
6. You’ll be able to see that the default tier is listed as the same type inherited from the storage account.
7. Now open up the drop-down menu and select the new storage tier from the list. In this example, we are going to change the tier of the blob from Cool to Archive tier.
8. While changing to Archive tier, you will get a warning message telling you that the blob will be inaccessible until it is rehydrated back to Hot or Cool tier and the process could take several hours. Click on “Ok” to switch the tier.
9. You can confirm the tier has been changed by accessing the properties of the blob. Go to Storage account->Blob service->Blobs-><container name> and you’ll see that the access tier is now listed as Archive.
And that’s it. You have now seen how to successfully change the default storage tier of your Azure account as well as how to change them for individual blobs on demand.
Azure storage has a lifecycle management capability, and using it data can be transitioned to lower-access tiers automatically based on pre-configured rule-based policies. This function also allows data to be deleted at the end of its lifecycle.
Using policies, data can be transitioned from Hot to Cool Tier, and from Cool to the Archive Tier or Hot to Archive Tier. Rules can be defined to be executed against the storage account once per day. Specific blobs and containers can be targeted using Prefix filters.
1. First we’ll start by configuring the Lifecycle management rules for a storage account. Open the storage account from Azure portal, browse to Blob service->Lifecycle Management, and click on “Add rule.”
2. Configure the rule to define the access tier management. In the Action set, define the blob lifecycle policy.
In this example, data is moved to Cool storage if it is not modified for 20 days and moved to Archive storage if not modified for 90 days. Click on “Next:Filter set” to configure the filter set.
Optionally, you can define a path to apply the rule to a specific container or subset of virtual folders. It will be used as a Prefix match while applying the rules.
3. Click on “Next:Review+Add” to validate the configuration. Next, click “Add” to complete the policy creation.
4. The rule will now be shown as enabled in the portal.
5. The steps described above are the list view configuration option for policies. The configuration can also be updated from the “code view” window from Storage->Lifecycle Management->Code view, as shown below.
You have now set up a data lifecycle policy rule that will automatically tier the data as reflected by your usage patterns.
NetApp Cloud Volumes ONTAP extends the data management capabilities of NetApp’s trusted ONTAP platform to Azure, enabling lower Azure storage costs with storage efficiencies. One such feature that adds to the efficiency of Azure storage usage is automated tiering between hot storage to cold storage tier in Azure.
Cloud Volumes ONTAP makes it possible to automatically tier between the Azure tiers: managed Azure SSD disk storage as performance tier and Azure Blob storage as capacity tier. This functionality is not available natively using the Azure native storage services.
Using this feature infrequently-accessed data is tiered from managed Azure SSD disk storage to the less-expensive blob storage, initially to the Hot Access tier, based on the tiering policy configured. In addition to that, customers can use the approach described in this blog to switch the Azure storage tier to cool tier for data that has not been accessed for 30 days to reduce the storage charged further. Cloud Volumes ONTAP makes this same functionality available for AWS storage tiers on Amazon EBS and Amazon S3.
Organizations can greatly benefit from the versatility of Cloud Volumes ONTAP combined with the cost benefits of Azure storage tiers for use cases such as LOB applications, backup, Disaster Recovery and more.