Storage
Mobility
Protection
Analysis and control
TOPICShare
About a decade ago, with the coming of the first AWS offerings, companies slowly shifted their IT infrastructures from being hosted entirely on-prem, passing through a middle point with hybrid architectures, to a point where they host their whole IT operations in the cloud. But not every company chooses to go all in with the cloud. Some want to keep just one foot in the water so to speak. For them, AWS approaches hybrid cloud management and all its different use cases through AWS Storage Gateway.
In this blog we are going to show you in a few steps how to set up a Storage Gateway through the AWS Console and start using the benefits of this hybrid cloud solution. For more details, such as Storage Gateway pricing, check out our previous blog post.
What Is Storage Gateway?
Storage Gateway is a hybrid deployment storage service on AWS that allows on-prem deployments to take advantage of AWS cloud services such as Amazon S3 and Amazon EBS. It also has storage management features that allow it to lower storage costs. Target use cases include file shares, migration, data tiering to the cloud, and more.
There are three different main gateway types and deployments:
- File Gateway: The Storage Gateway acts as a file server that delivers applications SMB and NFS protocol access while storing all the data in Amazon S3 and keeping all the file system structure.
- Volume Gateway: This gateway acts as an iSCSI target. There are two deployment options:
- Cached volumes keep a local copy of frequently accessed data
- Stored volumes are where the dataset resides locally in the gateway and frequent snapshots are taken and stored in S3 for possible recovery needs.
- Tape Gateway: This type of gateway acts as a VTL that interacts via iSCSI with your tape backup software, keeping the data locally and then asynchronously moving the tape backups to Amazon S3, Amazon S3 Glacier, and S3 Deep Archive.
Any of three gateway types above can be deployed in any of these forms:
- On-prem as a VM on ESXi or Microsoft Hyper-V.
- On-prem as a hardware appliance (which needs to be purchased).
- As a VM on VMWare Cloud in AWS.
- As an Amazon EC2 instance on AWS.
Note that Storage Gateway works as a hybrid solution exclusively using AWS cloud services.
Deploying Storage Gateway with EC2
Now let’s take a look at the specific steps that users can take when they are ready to get Storage Gateway set up as their hybrid deployment storage service.
Amazon EC2 Installation Example
In this example we are going to provide the steps to deploy the Gateway on an AWS EC2 instance using the AWS Management Console. Note that for the purposes of this article we are using the AWS console not physical equipment on-prem setups, although it is common for on-prem set-ups to be on top of a hypervisor or on the hardware appliance (purchased from Amazon).
1. After logging into the AWS Management Console, go to the Services section and select Storage Gateway. Make sure to select the correct AWS region you want this to be deployed in and then hit “Get Started.”
This will take you to the first step: Selecting the gateway type. Select the gateway type desired and then click “Next.” For this article, we are selecting a file gateway.
2. In the next screen, it’s time to select a host platform. This is where you decide whether to deploy the Storage Gateway as an EC2 instance, on a hypervisor, or on the hardware appliance.
In our example we will choose “Amazon EC2.” A set of recommendations are displayed at the bottom with the configuration settings generally needed for your gateway to run smoothly. Once you read these, click on “Launch Instance.” This will take you to the EC2 Console where you normally launch an EC2 instance.
3. Once you create the EC2 instance you will reach the Add Storage screen. Here, make sure to select “Add New Volume” as this is going to be needed for the gateway's local cache and upload buffer. You can turn to Amazon's recommendations to find the optimal cache and buffer disk sizes for your gateway.
4. In the next step you will configure your security group. This is another important aspect that enables access for all the expected protocol traffic.
Click “Review and Launch” to confirm the information that you previously entered, and then launch the instance.
6. You are now taken back to the Gateway console and the EC2 instance is set up. Click “Next” to proceed.
7. Now we will set up how your gateway is going to communicate with AWS services. You need a service endpoint for this which can be public with internet access or in a Virtual Private Cloud Endpoint.
If you choose to create the service endpoint in your VPC, all the traffic is going to stay within your VPC, even if it is generated on-prem and then passed to AWS. Follow AWS specifications for VPC endpoint deployments, especially for file gateways. In this example we will simply deploy it as a public endpoint.
8. Now you need to connect and activate your gateway. You must have access to the gateway from the browser used to access the Management Console.
If deploying on an EC2 instance, you can get the IP address you need to connect to from the instances console section, as shown in the image below. If deploying on a hypervisor, you can get the IP from the gateway local console.
Attaining the public IP address to activate the gateway from the EC2 console section.
Once you have the IP address, return to the Create Gateway prompt and input the address.
Inputting the IP address on the main Create Gateway screen.
9. If the connection is successful the console will take you to the Activation screen.
If activation is successful you will see the message that the Gateway is now active.
10. Now you need to configure the gateway's local disk for cache. Again, it is important to get a good estimate of what your cache and buffer needs will be.
Select “Configure Logging” to continue to the next step.
10. You can add optional logging to receive health notifications from AWS CloudWatch about your gateway's health status. Finally, you get to the screen showing you the gateway was created successfully.
Creating an NFS File Share
With the compute out of the way, it’s time to create storage for the gateway. To do that, follow the steps outlined below.
1. Go to the Storage Gateway Console and choose “Create File Share.” You need an existing Amazon S3 bucket to store the data. If you don't have one, go to the S3 Console and create one and then copy the name in the S3 bucket field.
2. In this step there are some settings you can choose from, including:
- The S3 Storage Class (S3-Standard, S3-Standard IA, S3 One Zone IA)
- Enabling guessing of MIME types for files uploaded to the bucket based on their extension (helps classifying the data)
- Whether the share is going to have cross-account access (if bucket is going to be accessed by users from other AWS accounts)
- The IAM role for the Storage Gateway for its access to the buckets. You can create your own IAM role and policy or let the Storage Gateway create one for you.
3. In the final step, you set access restrictions based on your IP address and mount options.
The file share will show as Available with all the mount instructions at the bottom.
So, let’s summarize what we did:
- Created and configured a Storage Gateway set up to run on an EC2 instance.
- Configured the endpoint connection.
- Activated the gateway and confirmed the local cache storage.
- Created an NFS file share and mounted it from the client.
Conclusion
Storage Gateway, a hybrid storage solution from Amazon, makes integration of on-prem data to the cloud a simple step, either for live applications using file or volume gateways or for backup data with EBS snapshots or Tape Libraries. Getting the Gateway set up and running is a simple procedure that can be done easily from the AWS Management Console. However, there are security and configuration details that need to be taken care of in order to set the best of possible configurations.
For another hybrid and multicloud data management solution, Cloud Volumes ONTAP can be used not only with AWS but with Azure and Google Cloud. On-prem ONTAP systems can be integrated with Cloud Volumes ONTAP nodes on AWS, Azure and Google Cloud through a single web-based interface called NetApp Cloud Manager, which can be used to configure and manage any hybrid and multicloud deployment.
Cloud Volumes ONTAP brings additional benefits to AWS including:
- Multiprotocol cloud file sharing access for SMB, NFS, and iSCSI.
- Data protection with NetApp Snapshot™ storage snapshot technology and disaster recovery.
- Advanced storage efficiencies such as thin provisioning, data compression, and deduplication, which can reduce cloud data storage costs and footprint by up to 70%.
- Intelligent data tiering between AWS storage tiers on high-performance block storage and low-cost object storage.
For more on hybrid storage management on AWS, check out this article on how Cloud Volumes ONTAP compares to Storage Gateway.
