Storage
Mobility
Protection
Analysis and control
TOPICShare
Get the most out of your Oracle database with Amazon Web Services (AWS) and the NetApp® Cloud Volumes Service.
Relational database systems are often the nerve center of an enterprise, standing behind financial transaction systems, automated teller machines (ATMs), and retail sales. As such, relational database systems have high performance requirements in addition to their other needs. In this blog post, I will attempt to guide you through the various storage options, revealing their performance capabilities along the way.
AWS is a solid platform for running relational databases, which traditionally scale up rather than out. The platform scales up in memory, CPU, and disk needs as load increases. Most of us are used to the concept of scaling up instance sizes to get more CPU and memory and even network, but what about disk? How far can storage go in the cloud to fit your application’s needs, and which technology should you use?
So, Mr. Owl, how many licks does it take to get to the center of a Tootsie Role Tootsie Pop? Wait a minute, wrong blog. Mr. Cloud Guy, how many disk operations does it take to saturate an Amazon EC2 instance? While not quite as tasty as “one, two, three, chomp,” the answer to the Amazon question is much more knowable.
EBS
Though the capabilities of the Amazon Elastic Block Store (EBS) drive types differ, the maximum capability of an Amazon Elastic Compute Cloud (Amazon EC2) instance is fixed. According to the AWS EBS user guide, at a maximum, no Amazon EC2 instance can generate more than 80,000 disk operations per second, as shown in the following table. I omitted the st1 and sc1 type EBS volumes from Amazon’s table, because I would not recommend what amounts to SATA drives for a relational database workload. Had I shown them, you would see the same 80,000 disk operations value.
The limit shown in the previous table refers to the maximum value attainable from the largest instance type. To understand how much I/O you can get from your specific instance type, refer to the AWS user guide for EBS-optimized instances—excerpted in the following table. This guide shows the maximum I/O count for any specific Amazon EC2 instance type. For example, according to the table, the c5.18xlarge is capable of generating 80,000 disk operations, whereas the c5.9xlarge can generate no more than 40,000 disk operations, and the c5.4xlarge on down can generate no more than 20,000 operations per second.
EBS and RDS
Although these EBS bandwidth limits are equally true for instances constituting the Amazon Relational Database Service (RDS), the disk I/O limits vary greatly. RDS instances configured with io1 devices can do no more than 40,000 disk operations per second versus the 80,000 maximum value specified in the previous table.
When configured with a gp2 block device, RDS instances are rate-limited at 160MBps. Amazon does not specify how many disk operations come with the gp2 volume, though it would be a safe bet that AWS is granting 3 IOPS per GB up to a maximum of 10,000 IOPS.
EFS
According to Amazon’s Elastic File System (EFS) limits guide, a single EFS export can support 7,000 file system operations per second using the standard configuration, and slightly more is attainable when the file system is configured in Max I/O mode. Amazon EFS thus leaves applications such as relational databases hungry for IOPS.
Amazon stipulates the following limits:
Cloud Volumes Service
The NetApp Cloud Volumes Service for AWS is uniquely positioned to capitalize on Oracle Direct NFS—Oracle’s take on the standard Linux NFS client. The Oracle Direct NFS client spawns many network sessions to each NetApp Cloud Volumes Service volume, doing so as load demands. The vast number of network sessions brings the potential for a significant amount of throughput for the database—far greater than a single network session can provide in AWS.NetApp Cloud Volumes Service, with Oracle Direct NFS, can take full advantage of Amazon EC2 front-end networking. Internal tests using the Oracle SLOB2 workload harness show that a single cloud volume can honor 144,000 file system operations per second on the c5.9xlarge instance type and 250,000 on the c5.18xlarge. At this point, folks, our documentation is not as good as that of our friends at AWS. Therefore, I’m left with having to show test results.
Test Results
The following graphs are the results of a 100% random read workload using the SLOB2 workload generator run against a database 12.2c that is run from a Red Hat Linux 7.4 Amazon EC2 instance.
The graph on the left shows just what the previously described data suggested: A c5.9xlarge instance can drive no more than 40,000 disk IOPS, whereas the c5.18xlarge supports no more than 80,000 disk IOPS. No surprises there. You’ll notice that the Oracle database reported excellent sub-millisecond latency by using io1 disks until the maximum disk I/O rate was met. Generally speaking, 2ms latencies or better are acceptable to database administrators (DBAs).
The graph on the right shows that Oracle is able to drive 250,000 file system IOPS at 2ms when using the c5.18xlarge instance, whereas Oracle can drive a NetApp Cloud Volume at 144,000 file system IOPS at below 2ms using the c5.9xlarge. I’ve heard but have not yet experienced for myself that the latency associated with Cloud Volumes Service for AWS varies from region to region, whereas EBS volume latency should not experience any such latency variances.
Test Prices
Sizing is not just about getting enough tech in place to meet business demands—in this case, storage. It is about just the right amount of a resource at the lowest possible price. Before closing, let’s explore the price of the storage used in this blog post. I created a 1TiB database, so let’s use that as the minimum amount of required storage. I will normalize the cost per operation because the Cloud Volumes Service outperforms the io1 device configuration.
There are some things you need to know before studying the next tables.
What you need to know about io1 sizing:
- AWS recommends io1 for databases rather than gp2—fitting my study.
- As you probably know, io1 devices are charged separately for I/O operations and capacity.
- No more than 50 IOPS can be allocated per io1 device per GiB of allocated capacity. You will notice that I had to assign more capacity to the io1 configuration in the c5.18xlarge scenario to maintain the ratio.
What you need to know about sizing with the NetApp Cloud Volumes Service for AWS:
- There are three service levels; I chose the extreme service level for this scenario, because it provides the highest I/O rate at the lowest cost per I/O operation.
- The three service levels map bandwidth to capacity; at the extreme service level, you get 128MB per TB of capacity (that is, 128KB per GB of allocated capacity).
- Whereas AWS uses the model of pay for IOPS, NetApp instead chose to base its billing model on paying for bandwidth—you can use the bandwidth however you wish. For example, 128KB of bandwidth can be consumed by two 64KB operations or sixteen 8KB operations.
- Although AWS and all the other hyperscalers use the base 2 definition for storage (KiB, MiB, GiB), the NetApp Cloud Volumes Service has gone with the base 10 definition for simplicity (KB, MB, GB). Personally, I’d rather see us use the base 2 definition as well—but I digress.
- The NetApp Cloud Volumes Service billing model works on 15-minute increments of granularity, by the way.
If I have lost you, or if you simply want to know more about the Cloud Volumes Service billing model, see the Cloud Volumes Service reference document.
In Closing
The cloud is all about scaling—be it scaling up as shown in this paper, or scaling out. Let your application needs drive your architectural decision. Sizing with EBS is straightforward and the latencies are consistently quite low. The EFS service is not generally a good fit for relational databases chiefly because of the EFS I/O limit of 7,000 IOPS. The RDS is simple to operate, but there is a cost in terms of disk I/O. The NetApp Cloud Volumes Service—a NoOps storage service that is the most straightforward of all—scales up the furthest of the four, though with slightly higher latency than EBS but much lower cost.
Come and check out NetApp Cloud Volumes Service for AWS for yourself. Visit NetApp Cloud Volumes Service for AWS.