Amazon EFS is an AWS file sharing service that lets you manage file shares, like those used on traditional networks, and mount them on Infrastructure as Service (IaaS) compute Instance or on-premises machines using the NFS protocol.
In this post, we’ll review the basics of AWS EFS, give you some tips to help you optimize your performance, and see how it compares to other AWS storage options. Finally, we’ll show you how NetApp Cloud Volumes ONTAP can help you get the most out of whatever file service option you choose.
In this article, you will learn:
AWS Elastic File System (EFS) is one of three main storage services offered by Amazon. It is a scalable, cloud-based file system for Linux-based applications and workloads that can be used in combination with AWS cloud services and on-premise resources. EFS offers a choice between two storage classes, Infrequent Access and Standard access, depending on your needs. Standard access storage is designed for frequently accessed files while Infrequent Access is intended for storing long-lived but less used files at a lower cost.
EFS uses the NFSv4 protocol for its file system structure, which mirrors a standard on-premise structure and simplifies transferring and accessing your files. It can be used in combination with Elastic Cloud Compute (EC2) instances or as a stand-alone file system. EFS does not require storage provisioning and is pay-for-use allowing you to scale services as needed.
Some of EFS’ key features include:
Shared Storage
Files are accessible from AWS services as well as from on-premise. They can be simultaneously accessed by up to a thousand EC2 instances within the cloud or via VPN or AWS Direct Connect, making EFS good for hybrid solutions. File access is possible across multiple AWS Availability Zones (AZs) and Regions, for easier collaboration and global remote work.
Scalable Performance
EFS is designed for low latency with IOPS and throughput that scale with usage and the number of attached instances, meaning that as storage size grows, performance increases. At peak performance, it offers 10 GB/sec throughput and 500k IOPS. EFS scales automatically as data is moved in or out, minimizing fears of running out of space or paying for storage you aren’t using. Learn more about how file sharing impacts cloud scalability and agility.
Secure and Compliant
EFS allows multiple layers of security and relies on your existing security infrastructure. It can be used with Amazon's Identity and Access Management (IAM) roles as well as VPC security groups and allows you to define individual file permissions using POSIX. EFS has built-in compliance with common regulatory standards, including PCI DSS, HIPAA, and SOC with the ability to meet others if necessary. Learn more about best practices for secure cloud file sharing.
Since a storage solution is only really useful when it can be backed up, you should be aware of your backup options with EFS. Keep in mind, other ways exist if you want to use third-party products integrated with AWS. Also, learn more about general considerations for backup and archiving in the cloud.
Amazon Backup Service
AWS Backup is a fully managed service that allows you to create, manage, and automate incremental backups according to a schedule you define through a central location. This system is PCI and ISO compliant and HIPAA eligible, to ensure that your compliance needs are covered. It is possible to use AWS Backup whether your system has a cloud-native, hybrid, or on-premise configuration.
This solution is easy to implement, and incremental backups help keep your costs low but requires manually pausing the applications and processes being backed up and only allows backups to be stored on EFS.
EFS to EFS Backup
There is no built-in EFS backup, and EFS does not have a native snapshot mechanism. So, before AWS Backup was released, backups had to be done using a template in AWS CloudFormation. This involves using scripts to access the AWS Data Pipeline, from which you must transfer data between multiple services before finally storing the backed-up data in EFS. With this process, you are still able to control backup schedules and life cycles as with the AWS Backup Service.
The main downsides of this option are that it does require some programming knowledge and is not easily monitored. Additionally, if you are not careful to change time constraints according to the amount of data you are backing up, your process can fail.
Backing Up to Amazon S3
Backing up your data to S3 is another option you might consider and one that can help decrease your storage costs. This process begins the same as the EFS to EFS backup but rather than moving the incremental backups from S3 to EFS at the end of the process, they are simply left in S3.
Like EFS to EFS, you can schedule your backups and manage life cycle length but you must also have some programming ability and be familiar with the use of scripts for automating processes.
To get the greatest ROI with EFS, you will need to take steps to optimize your configuration performance. Although specific changes will depend on your needs, the following tips apply to most EFS users.
Monitor Your EFS Burst Credits
One of the biggest shocks for EFS adopters comes the first time they run out of burst credits and they see their performance sharply drop. To avoid this, make sure that you monitor your burst credits either manually or by adding alerts in CloudWatch that will notify you when they run low. Making sure to schedule backups during periods of low use or during non-working hours will help save your credits for when they are truly needed.
You should keep in mind that EFS volumes start with a .5MB/s transfer rate and only 7.2 minutes of burst credits for throughput up to 100MB/s, which is probably not enough to meet your needs. You can increase these numbers by increasing the amount of data you’re storing but there is no way to simply buy additional credits.
Don’t Run Applications from EFS
If you plan to run and host applications from EFS, Elastic Block Storage (EBS) is a better option for you. EFS is not meant for the large file read volumes or fast speed that tasks like managing codebases or application deployment require, so you should not try to use it for these tasks. Instead, stick to tools that deploy code to local filesystems or containers.
Using EFS as it is designed, for storage of media assets, exported data files, asynchronous logs, etc. will ensure that you see the sort of performance you’re expecting and that your productivity doesn’t suffer.
Choose the Right Performance Mode
EFS is designed with two different performance mode options: General Purpose and Max I/O.
General Purpose is the default setting and the one that will work best for most users. It focuses on lower throughput levels in exchange for lower latency and is good for tasks like web-hosting and content management.
Max I/O focuses on higher throughput at the cost of higher latency, making it better for tasks such as big data analysis or media processing.
Keep in mind, both will see an increase in throughput performance as you attach more concurrent instances as long as the workload can be spread across the instances.
To know whether EFS truly is the best option for you, you’ll want to compare it to your other two main options, EBS and S3.
AWS EFS
EFS is fairly easy to learn to use because it mirrors a traditional file hierarchy structure. Since it allows multiple instances to be connected simultaneously across regions and AZs, it does not require as much redundant storage as other options and allows for easy cloud file sharing and collaboration.
As mentioned, EFS is ideal for use in global content management systems, big data analytics, and media processing workflows.
AWS EBS
EBS uses volumes to provide low-latency block storage for use with EC2 instances. It can be a bit complicated to set up as it requires that you choose your volume type, according to performance requirements, from the start and it doesn’t automatically scale like other options.
EBS is designed for application hosting and storage and high-volume databases. Learn more about AWS EBS in our extensive series of blog posts.
AWS S3
Simple Storage Service (S3) provides object storage, without the use of EC2 instances, that is accessible directly through the internet. It is the cheapest option and provides a lot of flexibility in storage, but can require some programming proficiency as it is typically managed through AWS Software Development Kits (SDKs). Learn how to mount Amazon S3 as a drive to simplify management.
S3 works well for static website hosting, version management, log file storage, media distribution or streaming, and big data analytics.
For a side-by-side comparison of these three options:
|
|
EFS |
EBS |
S3 |
|
Cost* |
$0.3 per GB/month and $6 per MB/month for provisioned throughput |
$.045-$0.125 per GB/month, depending on volume type |
$0.021-0.023 per GB/month depending on total storage |
|
Performance |
Up to 500k IOPS and 10+ GB/s throughput
Minimum of 3GB/s |
Varies by volume type
Minimum of 3 IOPS per GB of storage |
3500 PUT/LIST/DELETE requests per second
5500 GET requests per second |
|
Availability and Accessibility |
Highly available but no SLA defining percent
Accessed by up to 1K instances from multiple AZ or Regions |
99.99% available
Accessed via a single EC2 instance |
99.99% available
Accessed via Internet and API |
|
Access Control |
Managed via user and group level permissions File access requires endpoint access |
Managed via IAM and security groups |
Managed via IAM and bucket policies |
|
Storage and File Limits |
Unlimited storage size 47.9 TB file size |
16 TB storage per volume with unlimited volumes File size limited by volume size |
Unlimited storage size 5TB file size limit |
*AWS pricing is rather complex but the NetApp AWS calculator can help you better understand and reduce your costs.
NetApp Cloud Volumes ONTAP, the leading enterprise-grade storage management solution, delivers secure, proven storage management services on AWS, Azure and Google Cloud. Cloud Volumes ONTAP supports up to a capacity of 368TB, and supports various use cases such as file services, databases, DevOps or any other enterprise workload.
Cloud Volumes ONTAP provides a cloud file share service that supports efficient NFS and SMB file storage and facilitates secure cloud file sharing.
Shared storage enables you to store, distribute, and share your data across multiple locations and users. AWS offers a wide range of shared options for storage, including shared files, block, and object storage.
This article explains how shared storage works on AWS, and the main differences between AWS shared storage options.
Read more: AWS Shared Storage for Files, Block & Object Storage
AWS EFS is a popular cloud file sharing service that can be mounted on Amazon EC2 instances, alongside Amazon EBS.
Learn which scenarios EFS can and cannot provide high performance, understand EFS performance options and get performance optimization tips.
Read more: AWS EFS Performance Do’s and Dont's
Network File System (NFS) is a distributed file system protocol that lets users access files over a network similar to the way they access local storage.
In this article, we’ll look at 5 key considerations to address when using AWS EFS as your NFS solution, including operating system support, on-premise access to files, and managing backups.
Read more in AWS NFS File Shares with Amazon EFS: 5 Key Considerations
AWS EFS is Amazon’s fully managed cloud file sharing solution which supports the NFS protocol. Before selecting a file service it’s important to learn about the available options to see which solution can best meet your requirements.
In this article we take a deep dive into AWS EFS, examining its capabilities, storage capacity, costs and limitations. We take our analysis deeper by comparing the file service to Cloud Volumes ONTAP, another file sharing solution that comes with built-in storage efficiencies, cloning, tiering to Amazon S3 and cost-efficient snapshots.
Read more: AWS File Sharing with AWS EFS
The cloud doesn’t do everything for you: While some services have internal backup mechanisms, others do not, meaning that it becomes a responsibility that falls exclusively to the customer. That’s the case with NFS file share data stored on Amazon EFS. If you’re using Amazon EFS, you’ll need to make sure that you have an efficient Amazon EFS backup plan in place.
This article will take a look at a number of different Amazon EFS backup methods, including a new AWS tool called AWS Backup Service, performing an EFS to EFS backup, backing up to Amazon S3, using Terraform, and NetApp’s alternative: file share storage services with Cloud Volumes ONTAP.
Read more: EFS Backup: Backing Up NFS Data in AWS.
Across industries, most companies have a 80/20 split between the amount of data that is inactive over active. The same is true with file share data stored on services such as AWS EFS. To reduce the cost of storing this large amount of infrequently accessed files, AWS has created a new storage class for AWS EFS, called EFS Infrequent Access (EFS IA).
EFS IA effectively acts as a form of AWS tiered storage. What are the benefits of tiering file share data from EFS Standard to EFS IA? And how does this new solution from AWS compare to keeping file share data on Amazon EBS with Cloud Volumes, which leverages automated data tiering to Amazon S3? Find out in this post.
Read more: EFS IA Vs. Cloud Volumes ONTAP Data Tiering: File Share Tiering Options on AWS.
Cloud file storage refers to the storage of digital files on remote servers that are accessed through the internet. AWS (Amazon Web Services) offers several file storage services that allow you to store and manage files in the cloud. Each AWS file storage service is designed for specific use cases and provides different features, performance characteristics, and pricing models.
Read more: 4 AWS File Storage Solutions and How to Make the Most of Them.
Together with our content partners, we have authored in-depth guides on several other topics that can also be useful as you explore the world of IaaS.
Authored by NetApp
Authored by NetApp
Authored by Spot