Amazon Web Services (AWS) gives its users a lot of choices when it comes to finding the perfect storage plan for cost and performance optimization. One of the most popular S3 storage classes is S3 Intelligent-Tiering, which has helped customers save $750 million in storage costs since its launch.
S3 Intelligent-Tiering enables the optimization of storage costs by moving objects between different storage tiers depending on the frequency of access. With AWS’s recent launch of the Archive Instant Access tier, the Intelligent-Tiering can now automatically archive data for cost benefits, while offering sub-millisecond retrieval access.
In this article, we learn about S3 Intelligent-Tiering’s Archive Instant Access tier, its benefits, and various use cases.
Amazon S3 Intelligent-Tiering is a cloud-based, object storage class that automatically archives objects to cost-efficient storage tiers for maximum benefit. With its intelligent algorithm, Intelligent-Tiering automatically helps optimize your storage costs without sacrificing performance or availability. Although the storage class is ideal for data that has unknown or changing data usage patterns, such as application logs, media files, and sensor data, it can also be used for frequently accessed data that doesn't need the lowest latency possible, such as backups and disaster recovery files.
S3 Intelligent-Tiering continuously monitors your data usage and automatically moves it between the three tiers—hot, warm, and cold—based on its access patterns. The hot tier is optimized for frequently accessed data, while the warm and cold tiers are designed for infrequently and rarely accessed data. As one of its key benefits, you only pay for the storage that you consume and a small monthly fee for monitoring and automation.
The Archive Instant Access (AIA) storage tier provides fast retrieval times for infrequently accessed data, while still providing the cost benefits of storage in the cloud. Compared to the Infrequent Access (IA) tier, the Archive Instant Access tier enables customers to archive data without compromising on availability or durability, knowing that they can instantly retrieve the data they need, without having to first restore it from a lower tier.
The AIA tier is auto-enabled for all storage objects in the S3 Intelligent-Tiering class, where any data object in IA tier not accessed for 90 days is automatically moved to the AIA tier. The storage tier also offers the flexibility to transition data back to the Standard or Bulk storage tiers if needed, without incurring upfront investment.
The benefits of using the S3 Intelligent-Tiering AIA tier include:
No retrieval charges
While Amazon S3 pays for the data egress from the archive tier, you only pay for what you use. As a result, there are no charges incurred towards reconstruction or retrieval of data archived in the AIA tier. This makes it an ideal storage solution for infrequently accessed data that still needs to be quickly accessible, without incurring additional charges.
No minimum storage duration
There is no minimum storage duration for S3 Intelligent-Tiering AIA tier. You can store objects in this tier for as short or as long as you want, without having to worry about any minimums. This makes it ideal for storing data that you may not need immediate access to, but still want to retain it for future use. You only pay for the storage you use, so you can minimize your costs by only storing data that you'll need in the future.
Zero operational overhead
The algorithm that powers S3 Intelligent-Tiering uses machine learning to monitor access patterns and make real-time decisions on identifying the best storage tier for each object. This means that you no longer have to worry about manually moving objects between storage tiers or paying for unnecessary storage. This is in contrast to other storage solutions that require administrator inputs and regular tuning to maintain optimal performance and cost efficiency.
Plus, the AIA tier provides immediate accessibility of your archived data by reducing the efforts involved in moving objects across tiers before retrieval.
Low latency and high throughput
The AIA tier relies on S3’s foundational techniques, including data localization, multiple availability zones, optimized file system, and efficient caching mechanisms to reduce latency. This design pattern enables the storage tier to offer sub-millisecond retrieval even for rarely accessed data.
In addition, the storage tier also uses the optimum mix of hard disk drives (HDDs) and solid-state drives (SSDs) to achieve high throughput. This makes it perfect for workloads that rely on high performance storage, such as video streaming or big data analytics.
Amazon S3's Intelligent-Tiering AIA tier makes the most sense for data that is infrequently accessed but still needs to be quickly available when needed. This includes data such as backups, log files, and media files. The Archive tier has a lower per-gigabyte price than the Standard storage class, making it more cost-effective for long-term storage. And, because it's still in S3, you can take advantage of all the benefits that come with using S3, like security and durability.
Some appropriate use cases of AIA tier include:
Infrequently Accessed Archive Storage
One common use case for the AIA tier is storing data that is infrequently accessed. The low cost of storage and retrieval makes this tier ideal for archived data, as it can be stored long-term without incurring high costs. Additionally, the high durability of the tier means that data will be safe and accessible even if it is rarely accessed.
Quick Access Storage Tier
Another common use case for the AIA tier is storing data that can be accessed quickly in the event that it should ever be needed. The key benefit of using this tier for this type of data is its quick access time. This means that users can access the data they need without having to wait for it to be retrieved from slower forms of storage. Additionally, the high durability of the tier means that data will be safe and accessible even if it is accessed frequently.
Ideal for Unpredictable Access Patterns
The AIA tier is ideal for storage objects whose access patterns vary unpredictably over time. For instance, in a music record collection and streaming application, users often access music that has been recently uploaded. The streaming trends gradually reduce as the record grows older. After a few months of no access, S3’s Intelligent-Tiering can move the audio files to the AIA tier. A sudden surge in the artist’s popularity may result in growing interests toward their older music, data of which can then be retrieved instantly for streaming through the AIA tier or can be migrated over to a Frequent Access tier based on the access pattern.
To get started with Intelligent-Tiering, you first need to create a new storage class. This can be done by using the S3 Console, AWS CLI, or the PUT API object. Once you've created the storage class, you can then move your data into it using a simple API call.
Note: S3 Intelligent-Tiering now automatically tiers any object in the IA tier that hasn’t been accessed in 90 days to the AIA tier. In order to manually configure the rules towards individual objects of a bucket, you can follow the steps below.
Before you apply tiering policies at the object level of a bucket, navigate to the S3 console and select the bucket you want to enable Intelligent-Tiering on. Then, select the Management tab and click on Intelligent-Tiering. From there, you can make sure by checking if Automatic or Manual mode is enabled. With automatic mode, S3 will automatically move objects between the tiers based on usage patterns. Using manual mode allows you to specify which tier an object should be stored in.
Let us assume, you intend to apply tiering rules on individual objects of an existing bucket. To achieve this, follow the workflow below:
You can also use AWS CLI commands to manage configurations of the S3 Intelligent tier.
A typical approach of doing so is to use the put-bucket-intelligent-tiering-configuration command and specify the configuration as a JSON file to enable different tiering options. The following command assigns the AIA configuration to the darwinbucketexample bucket.$ put-bucket-intelligent-tiering-configuration
{
"Id": "darwinbucketexample",
"Filter": {
"Prefix": "filter-criteria1",
"Tag": {
"Key": "object-tag-key1",
"Value": "object-tag1"
},
"And": {
"Prefix": "filter-criteria2",
"Tags": [
{
"Key": "object-tag-key2",
"Value": "object-tag2"
}
...
]
}
},
"Status": "Disabled"
"Tierings": [
{
"Days": integer,
"AccessTier": "ARCHIVE_ACCESS"|"DEEP_ARCHIVE_ACCESS"
}
...
]
}
You can also use the PUT API object to apply rules on how data is moved to S3 Intelligent-Tiering, and is then archived.
For example, to move the darwinbucketexample bucket to Intelligent-Tiering, you will have to include INTELLIGENT_TIERING as a storage class under the x-amz-storage-class header.
PUT /darwin-image.jpg HTTP/1.1
Host: darwinbucketexample.s3.<Region>.amazonaws.com (http://amazonaws.com/)
Date: Thu, 22 Dec 2021 15:50:33 GMT
Authorization: authorization string
Content-Type: image/jpeg
Content-Length: 22452
Expect: 100-continue
x-amz-storage-class: INTELLIGENT_TIERING
Once done, you can use the PutBucketIntelligentTieringConfiguration API operation to configure the archive rules of the Intelligent-Tiering for a specific bucket or bucket-object.
While S3 Intelligent-Tiering adds a much needed feature for S3 users, NetApp BlueXP Cloud Volumes ONTAP can take it further as a data management layer for AWS.
Cloud Volumes ONTAP data tiering makes it possible to automatically tier infrequently used EBS storage data to more cost effective S3 Intelligent-Tiering and move it back automatically when required. Tiering between object and block storage isn’t possible on AWS natively.
But that’s not where the benefits end. Cloud Volumes ONTAP gives AWS users higher levels of data protection using more cost- and space-efficient NetApp Snapshot technology and Cloud Backup. Built-in storage efficiencies—including deduplication, compression, and thin provisioning—can lower overall storage costs, not just for S3 but also for more-expensive AWS EBS storage. And BlueXP replication with SnapMirror® makes it possible to move your data between AWS regions, on-prem deployments, and even different clouds easily and efficiently.
Learn more about how Cloud Volumes ONTAP can help with these Cloud Volumes ONTAP Storage Tiering Case Studies.
To enable Intelligent-Tiering on an S3 bucket, start by choosing the name of the bucket from the bucket list. On the bucket’s page, select Properties, scroll to the S3 Intelligent-Tiering Configurations section, and then click Create. Additional details on configuration settings of intelligent-tier for an S3 bucket can be found on the AWS official documentation.
Although S3 Intelligent-Tiering may incur higher initial cost, it can ultimately save organizations money in the long run. With S3 Intelligent-Tiering, data is automatically moved to the most cost-effective storage tier, which can save them money on storage costs. This is typically achieved by automatically storing data in the most appropriate storage class based on access patterns, making it ideal for infrequently accessed data. Additionally, S3 Intelligent-Tiering provides added protection against data loss and corruption, which can further reduce cost contingency.