Storage
Mobility
Protection
Analysis and control
TOPICShare
Organizations that wish to combine their on-premise storage systems with cloud services must ensure that their storage is compatible with the cloud environment. For those who use the Amazon cloud, the answer is the AWS Storage Gateway, which enables a hybrid cloud deployment with seamless integration between the cloud and on-premise, offline AWS-compatible storage. The Storage Gateway is often an essential component of an AWS Migration strategy.
In this article, you will learn:
- What Is AWS Storage Gateway?
- AWS Storage Gateway Types and Architecture Options
- AWS Storage Gateway Pricing
- Optimizing AWS Storage Gateway Performance
- Cloud Volumes ONTAP for AWS Migration and Hybrid Storage
What Is AWS Storage Gateway?
Amazon Storage Gateway is a storage solution that you deploy on-premises, and allows you to integrate Amazon cloud storage with on-premise storage systems, creating a hybrid storage environment.
You can deploy Storage Gateway and allow on-premise systems to access Amazon services like S3, Glacier, and EBS. It can also allow Amazon services to access on-premise resources, so that Amazon services for machine learning, monitoring, logging, etc. can be applied to data that is stored on-premises.
AWS Storage Gateway uses standard protocols, including NFS, SMB and iSCSI, so it allows on-premise systems to access cloud resources seamlessly, just as they would access on-premise storage.
A key benefit of Storage Gateway is that it uses local caching to provide frequently-used cloud data to on-premise systems, without the high latency of accessing the public cloud.
AWS Storage Gateway Types and Architecture Options
AWS Storage Gateway lets you connect local storage to cloud storage in three ways: by saving individual files to cloud storage, by mounting a cached or stored volume as a local drive, or by exposing cloud storage as a tape interface, which can connect to legacy backup systems.
The following sections show three AWS Storage Gateway architectures supporting these three use cases.
File Gateway—Local Access to S3 for Files
AWS File Gateway lets you store local files as objects in an S3 bucket, creating an S3 storage gateway. S3 buckets can be mounted locally as Network File System (NFS) drives, or you can store and retrieve files directly using the SMB protocol.
You can store files to S3 Standard, S3 Infrequent Access and S3 One Zone IA. File ownership and operation timestamps are stored in the user-metadata of the relevant objects in S3.
Source: Amazon Web Services
Volume Gateway—Local Access to S3 and EBS for Disk Volumes
The Volume Gateway provides access to Amazon EBS volumes, using the iSCSI block protocol. There are two ways of working with Volume Gateway:
- Cached Volumes—stores frequently accessed data locally using EBS, with Amazon S3 as main data storage. You can create a storage volume of up to 32TB and connect it to local systems as an iSCSI device. The storage volume then stores any infrequently-used data automatically to S3, reducing the load on local store, while using cloud storage in a cost-effective manner.
- Stored Volumes—treats your local storage as main data storage, but performs asynchronous backup to Amazon S3, as an EBS snapshot. This gives you an off-site backup of a local drive. The Stored Volume is mounted as an iSCSI device within on-premise applications. Stored Volumes can be 1GB to 16TB in size.
Source: YouTube, by Amazon Web Services
Tape Gateway
The Tape Gateway uses a Virtual Tape Library (VTL) interface, and lets you store data on virtual tapes, with the actual data stored in Amazon S3, Amazon S3 Glacier, or Amazon S3 Glacier Deep Archive. The idea is to replace physical tapes with a compatible interface that lets you store data on Amazon.
You can connect the Tape Gateway to existing backup devices using the iSCSI protocol, maintaining existing backup workflows and systems. Tape Gateway lets you save data directly to your S3 buckets, or use third-party backup services like NetBackup and Veeam.
Source: Amazon Web Services
AWS Storage Gateway Pricing
AWS has differential pricing depending on the type of gateway you use—file gateway, volume gateway or tape gateway. You can also use an AWS Calculator to help you plan your overall budget. Prices we mention in this section are subject to change—see the official pricing page for up-to-date prices.
File Gateway Pricing
Any data you store on Amazon S3 via the file gateway is priced using standard S3 pricing. In addition, you pay $0.01 per GB written to cloud storage by your local storage gateway.
Volume Gateway Pricing
EBS snapshots created from your Stored Volumes are billed using standard EBS pricing. In addition:
- Volume storage is billed at $0.023 per GB-month
- Data written to cloud storage by your gateway is billed at $0.01 per GB
Tape Gateway Pricing
The tape gateway is priced differently depending on the S3 storage tier you use for your virtual tapes:
- $0.023 per GB-month for S3
- $0.004 per GB-month for S3 Glacier
- $0.00099 per GB-month for S3 Glacier Deep Archiv
In addition, requests are billed as follows:
-
$0.01 per GB for data written to AWS storage from the local gateway
-
$0.01 per GB for tape retrieval from S3 Glacier
-
$0.02 per GB for tape retrieval from S3 Glacier Deep Archive
-
$0.0032 per GB for moving a virtual tape from S3 Glacier to Deep Archive
Optimizing AWS Storage Gateway Performance
Here are three ways you can improve AWS Storage Gateway performance.
Use High-Performance Disks
You can optimize Storage Gateway performance by adding high-performance disks, such as solid state drives (SSDs) and NVMe controllers. Instead of using Microsoft Hyper-V NTFS, prefer to attach virtual disks directly to VMs on a storage area network (SAN).
In general, increasing disk performance can increase throughput and increase I/O operations per second (IOPS) for Storage Gateway.
Use Separate Physical Disks for Gateway and Cache
When setting up a gateway disk, do not use the same underlying physical disk disk for the upload buffer and cache storage. For example, in VMware ESXi, the default physical storage resource is represented as a virtual disk. When deploying a gateway VM, you select a data store to store the VM files. Make sure that the virtual disk used for the gateway VM is on a separate physical disk than the virtual disk used for the buffer and cache.
Use a Larger Block Size for Tape Drives
For tape gateways, the default block size for tape drives is 64 KB. However, you can increase the block size up to 1MB to improve I/O performance.
The block size you choose depends on the maximum block size supported by your backup software. It is recommended that the size of the backup software tape drive block be set to the largest possible size. However, this cannot exceed the maximum 1MB supported by Storage Gateway.
Using VMware vSphere High Availability with AWS Storage Gateway
If you run VMware vSphere on your on-premises data center, you can use AWS Storage Gateway’s VMware integration to enable high availability. VMware vSphere can provide a series of application-level health checks via vSphere High Availability (VMware HA). This approach helps protect storage workloads from hardware, hypervisor, or network failures. It also helps avoid connection timeouts and software errors like file share unavailability.
With this integration, gateways deployed in your local VMware environment or VMware Cloud on AWS can automatically recover from most service outages. Usually this is done within 60 seconds and no data is lost.
Cloud Volumes ONTAP for AWS Migration and Hybrid Storage
NetApp Cloud Volumes ONTAP, the leading enterprise-grade storage management solution, delivers secure, proven storage management services on AWS, Azure and Google Cloud. Cloud Volumes ONTAP supports up to a capacity of 368TB, and supports various use cases such as file services, databases, DevOps or any other enterprise workload.
Cloud Volumes ONTAP utilizes NetApp SnapMirror® technology, to migrate, replicate, and synchronize data between on-premise and AWS.
NetApp’s Cloud Manager and Cloud Volumes ONTAP lets you access and manage data seamlessly across hybrid environments. With Cloud Volumes ONTAP, you don’t need to worry about the physical placement of data. It can be made accessible anywhere in the hybrid architecture per application requirements.
