Storage
Mobility
Protection
Analysis and control
TOPICShare
One of our clients had their website running on Wordpress using an IIS server for years. However, a rise of traffic and a need for scale convinced them that moving to the cloud would be a much better option.
They explored a number of different public cloud providers, but ultimately the market success of AWS and its wide range of services — specifically their storage offerings — led the company to choose AWS.
During the migration study, engineers suggested that most of the pages of the site were static and therefore not subject to frequent changes. That meant that hosting on Amazon S3 as a static website would be the most cost-effective and manageable storage for those pages.
As it turns out, Amazon S3 was the most appropriate hosting solution for the website’s static content. However, they encountered a number of issues during the migration which they did not expect. In this article we will highlight a few such issues to consider before migrating to Amazon S3.
Cloud Data Migration Challenges
1. Amazon S3 Bucket Name Restrictions
An Amazon S3 bucket name has certain restrictions. The biggest of these Amazon S3 bucket name restrictions is that every bucket name used on AWS has to be unique. For hosting a static website, it is mandatory for a bucket name to be the same as the DNS.
For example, if your website is hosted as “yourwebsitesnamehere.com” then the bucket name should be “yourwebsitesnamehere.com.” Here it is very important that the same bucket name is available and that it complies with Amazon S3 naming conventions.
2. AWS SSL Certificates and Amazon CloudFront
It is recommended to have an AWS SSL certificate for a website. It will help for better indexing for search engines as well as for establishing the reliability of your site. Unfortunately, Amazon S3 does not provide an option for SSL with static website hosting. Although an individual object can be accessed using “https,” when it is hosted as a website you cannot access it with SSL. The best solution to this problem is to use Amazon CloudFront.
Amazon CloudFront is a content delivery network (CDN) offered by AWS that is best suited for distributing Amazon S3 objects (or content) from various Amazon CloudFront edge locations. You can use Amazon CloudFront to distribute Amazon S3 bucket content and enable SSL. The best part is AWS Certificate Manager, which offers free AWS SSL certificates to be used with AWS services. With a few clicks you can make sure that all your content will be distributed by Amazon CloudFront using https only.
Now, just configure your CloudFront distribution with Amazon Route 53 and you are all set to use your website using SSL.
3. Defining Cache Policy with Amazon CloudFront
Although Amazon CloudFront helps implement https protocols, it also comes with one consideration. Amazon CloudFront is ultimately a caching service that caches the content at edge locations. It is very important that you define cache policy the best way possible.
In some cases when a cache is not expired, your new updates might not be visible on the site. In this case you might need to invalidate all of the cache to make the updated page available to site visitors.
Be specific when invalidating: if you provide generic criteria it will clear all objects including those not modified, too.
4. Data Consistency
One thing to remember while uploading objects to Amazon S3 is its data consistency models. When you upload new objects, they are available immediately but when you update or delete objects that data might not be available due to Amazon S3’s eventual consistency model.
When updated pages do not load, don’t assume that it is a cache issue with Amazon CloudFront, as mentioned above: it can also be an Amazon S3 data consistency issue.
5. Setting Up HTTP Headers
One of the bigger challenges in migrating from IIS to Amazon S3 in an AWS migration is that you can't set HTTP request/response headers on all migrated files in one go. Either you have to do it for individual files or you have to write a program to do it. Thankfully, there is AWS Lambda@Edge.
AWS announced this service in late 2016 with the purpose of handling requests and responses being processed through Amazon CloudFront.AWS Lambda@Edge also gives an added advantage to indexing for search engine results. When search engines crawl web pages, Amazon S3 bucket pages might not be crawlable due to security (such as AWS WAF/OAI, which we’ll discuss more below). With AWS Lambda@Edge you can add a default index page for each folder.
6. AWS S3 Security
With cloud and publicly-hosted websites, security is a prime concern. In an AWS migration, the SSL part will be taken care of with AWS Certificate Manager and Amazon CloudFront, however accessing buckets can be restricted with Origin Access Identity (OAI).
When using Amazon CloudFront you have the added advantage of being able to implement the AWS firewall, AWS WAF. AWS WAF allows you to control distribution requests based on conditions you set.
One more security option that works with Amazon S3 is setting proper bucket policy using Amazon S3 security access control lists (ACL). With OAI, you can always restrict bucket access, however it is also important that you use proper IAM policies to give access only to internal users. Refer to these AWS guidelines for more on securing Amazon S3 access.
7. Optimizing with Amazon S3 Storage Types
Migrating to Amazon S3 is a good way to save on the cost of storage, however it is important to use the right Amazon S3 storage type too. Amazon S3 offers three storage classes for static website hosting.
Each storage type has a different SLA and pricing mechanism. When your website’s static pages aren’t accessed frequently, it is advisable to use the Amazon S3 Infrequent Access object storage class to save on storage costs.
To make the most of an AWS migration, understand your usage pattern and use the right storage class, as SLA and consistency varies for different classes.
8. Other Solutions
In addition to the above challenges, when you have an extremely large number of files, there are some other AWS services available to help you migrate data to Amazon S3, such as AWS Snowball, AWS Snowmobile, AWS Storage Gateway, to name just a few. You need to ensure that data migration is over secure channel and reliable too.
Overall, Amazon S3 is a great tool for hosting static content, but as we just saw, there are a number of things to keep in mind when migrating your files there. Additional considerations are data persistence (making sure all files are migrated), security, and the migration performance when moving a large number of files.
Taking all those considerations together, it seems as if carrying out a cloud data migration on your own and configuring all the security, AWS migration, coding, and more can amount to a lot of time-consuming and costly effort.
To make migrations faster and to save expenses, there are tools available to help ease the process of migrating cloud data, namely Cloud Sync, NetApp’s solution for data migration to Amazon S3.
Migrating to Amazon S3 with Cloud Sync
Moving to the cloud takes time, and the more data you move, the more expensive it can be. Cloud Sync expedites data migration to Amazon S3 so your site migration is quicker. On top of this, Cloud Sync offers security and reliability for transferring and synchronizing data to Amazon S3 from NFS or CIFS file shares.
Cloud Sync is a versatile tool: it is capable of converting file-based NFS/CIFS data sets into the object format used by Amazon S3. In addition to the initial replication, Cloud Sync only replicates the incremental changes during subsequent syncs. Cloud Sync uses a Data Broker that can run on an Amazon EC2 instance or an on-premises VM to sync data to Amazon S3.
Cloud Sync is perfect for use cases such as cloud on-boarding, data replication, data migration, data archiving, data collaboration, and for data analytics in the cloud.
Final Note
Amazon S3 is an inexpensive, scalable and secure solution for static website hosting. However, you need to pay special attention to certain aspects of the AWS migration to avoid challenges and enable special configurations. Migrating your website or any kind of data can be a daunting task considering the amounts of data involved, and that data’s importance to the site and its reliability.
In these AWS migration scenarios, instead of using a DIY transfer tool, it is much more effective to use tools such as Cloud Sync for quicker and safer cloud data migration to Amazon S3.
For an enterprise this is a secure and reliable solution as it will reduce migration overheads.If you’re ready to migrate data to Amazon S3, but want to save on time and costs, try Cloud Sync today with this free 14-day trial.
