Organizations that wish to combine their on-premise storage systems with cloud services must ensure that their storage is compatible with the cloud environment. For those who use the Amazon cloud, the answer is the AWS Storage Gateway, which enables a hybrid cloud deployment with seamless integration between the cloud and on-premise, offline AWS-compatible storage. The Storage Gateway is often an essential component of an AWS Migration strategy.
In this article, you will learn:
Amazon Storage Gateway is a storage solution that you deploy on-premises, and allows you to integrate Amazon cloud storage with on-premise storage systems, creating a hybrid storage environment.
You can deploy Storage Gateway and allow on-premise systems to access Amazon services like S3, Glacier, and EBS. It can also allow Amazon services to access on-premise resources, so that Amazon services for machine learning, monitoring, logging, etc. can be applied to data that is stored on-premises.
AWS Storage Gateway uses standard protocols, including NFS, SMB and iSCSI, so it allows on-premise systems to access cloud resources seamlessly, just as they would access on-premise storage.
A key benefit of Storage Gateway is that it uses local caching to provide frequently-used cloud data to on-premise systems, without the high latency of accessing the public cloud.
AWS Storage Gateway lets you connect local storage to cloud storage in three ways: by saving individual files to cloud storage, by mounting a cached or stored volume as a local drive, or by exposing cloud storage as a tape interface, which can connect to legacy backup systems.
The following sections show three AWS Storage Gateway architectures supporting these three use cases.
AWS File Gateway lets you store local files as objects in an S3 bucket, creating an S3 storage gateway. S3 buckets can be mounted locally as Network File System (NFS) drives, or you can store and retrieve files directly using the SMB protocol.
You can store files to S3 Standard, S3 Infrequent Access and S3 One Zone IA. File ownership and operation timestamps are stored in the user-metadata of the relevant objects in S3.
Source: Amazon Web Services
The Volume Gateway provides access to Amazon EBS volumes, using the iSCSI block protocol. There are two ways of working with Volume Gateway:
The Tape Gateway uses a Virtual Tape Library (VTL) interface, and lets you store data on virtual tapes, with the actual data stored in Amazon S3, Amazon S3 Glacier, or Amazon S3 Glacier Deep Archive. The idea is to replace physical tapes with a compatible interface that lets you store data on Amazon.
You can connect the Tape Gateway to existing backup devices using the iSCSI protocol, maintaining existing backup workflows and systems. Tape Gateway lets you save data directly to your S3 buckets, or use third-party backup services like NetBackup and Veeam.
Source: Amazon Web Services
AWS has differential pricing depending on the type of gateway you use—file gateway, volume gateway or tape gateway. You can also use an AWS Calculator to help you plan your overall budget. Prices we mention in this section are subject to change—see the official pricing page for up-to-date prices.
File Gateway Pricing
Any data you store on Amazon S3 via the file gateway is priced using standard S3 pricing. In addition, you pay $0.01 per GB written to cloud storage by your local storage gateway.
Volume Gateway Pricing
EBS snapshots created from your Stored Volumes are billed using standard EBS pricing. In addition:
Tape Gateway Pricing
The tape gateway is priced differently depending on the S3 storage tier you use for your virtual tapes:
In addition, requests are billed as follows:
$0.01 per GB for data written to AWS storage from the local gateway
$0.01 per GB for tape retrieval from S3 Glacier
$0.02 per GB for tape retrieval from S3 Glacier Deep Archive
$0.0032 per GB for moving a virtual tape from S3 Glacier to Deep Archive
Here are three ways you can improve AWS Storage Gateway performance.
You can optimize Storage Gateway performance by adding high-performance disks, such as solid state drives (SSDs) and NVMe controllers. Instead of using Microsoft Hyper-V NTFS, prefer to attach virtual disks directly to VMs on a storage area network (SAN).
In general, increasing disk performance can increase throughput and increase I/O operations per second (IOPS) for Storage Gateway.
When setting up a gateway disk, do not use the same underlying physical disk disk for the upload buffer and cache storage. For example, in VMware ESXi, the default physical storage resource is represented as a virtual disk. When deploying a gateway VM, you select a data store to store the VM files. Make sure that the virtual disk used for the gateway VM is on a separate physical disk than the virtual disk used for the buffer and cache.
For tape gateways, the default block size for tape drives is 64 KB. However, you can increase the block size up to 1MB to improve I/O performance.
If you run VMware vSphere on your on-premises data center, you can use AWS Storage Gateway’s VMware integration to enable high availability. VMware vSphere can provide a series of application-level health checks via vSphere High Availability (VMware HA). This approach helps protect storage workloads from hardware, hypervisor, or network failures. It also helps avoid connection timeouts and software errors like file share unavailability.
With this integration, gateways deployed in your local VMware environment or VMware Cloud on AWS can automatically recover from most service outages. Usually this is done within 60 seconds and no data is lost.
NetApp Cloud Volumes ONTAP, the leading enterprise-grade storage management solution, delivers secure, proven storage management services on AWS, Azure and Google Cloud. Cloud Volumes ONTAP supports up to a capacity of 368TB, and supports various use cases such as file services, databases, DevOps or any other enterprise workload.
Cloud Volumes ONTAP utilizes NetApp SnapMirror® technology, to migrate, replicate, and synchronize data between on-premise and AWS.
NetApp’s Cloud Manager and Cloud Volumes ONTAP lets you access and manage data seamlessly across hybrid environments. With Cloud Volumes ONTAP, you don’t need to worry about the physical placement of data. It can be made accessible anywhere in the hybrid architecture per application requirements.