It was the management guru Peter Drucker who first coined the phrase, “You can’t manage what you can’t measure.”
His insight has inspired many companies to do a better job of tracking their key performance indicators, and to base their strategic business decisions on reliable metrics.
Drucker’s adage most certainly holds true for those tasked with managing their organization’s cloud costs. Without a clear understanding of which services are being consumed, how much is being used in each category, and how much you are being charged for that usage, it is very difficult—if not impossible—to manage those costs.
The good news for AWS customers is that AWS invoices provide a wealth of detailed information about AWS cloud usage and costs. Knowing how to extract that information from the invoices can often provide visibility into how usage and cost can be optimized, without compromising performance or quality of service.
Before delving into how the key AWS usage reports and invoices are structured, it is important to understand how AWS services are categorized and priced. Here are the 5 main AWS billing categories:
1. Compute and Networking:
This category includes services such as AWS EC2 (virtual servers in the AWS cloud); Elastic Load Balancing (distributes network traffic across the virtual servers); Auto Scaling (automatically scales virtual servers based on demand); and AWS ECS (provides Docker containers on AWS EC2 virtual servers).
2. Storage and Content Delivery:
This category includes services such as Amazon S3 (scalable storage in the AWS cloud); CloudFront (a global content delivery network); AWS EBS (storage volumes for EC2 virtual servers), and AWS Glacier (low-cost archival storage).
3. Database(s):
This category includes products and services such as managed relational (AWS RDS or Aurora) or NoSQL (DynamoDB) databases, data warehousing (AWS Redshift), and database migration services.
4. Code Deployment and Management:
This category includes services such as AWS CodeDeploy (automated code deployment), AWS CodeBuild (building and testing code), and AWS CodePipeline (continuous delivery of released software).
5. Application Services:
This category includes services such as AWS AppStream (which hosts streaming applications and streams input and output to user devices), AWS Elastic Transcoder (converts digital media into required end-device formats), AWS SQS (allows application components to store queued data for retrieval by other components), and AWS SWF (coordinates tasks across application components).
Each billable service has its own pricing structure. AWS EC2 On-Demand compute services, for example, are charged per virtual server instance-hour, calculated from the time an instance is launched until it is terminated or stopped.
In addition, there may be transfer costs for data transferred into or out of an AWS EC2 instance and those charges are calculated per volume in GBs. AWS Elastic Load Balancing is charged for each hour (or partial hour) that the load balancer is running and for each GB of data transferred through it.
Amazon S3 On-Demand storage services are charged per GB used, while AWS EBS provisioned storage services are charged per GBs/month. In addition, requests such as GET, PUT, POST, etc., are charged per a block number of requests, ranging from blocks of 1,000 up to 10,000. As in compute services, there may also be data transfer costs associated with storage services.
The AWS RDS On-Demand service is priced per hour per DB instance used. Its NoSQL DynamoDB, on the other hand, is priced on a monthly basis according to provisioned read and write capacity units, plus an hourly rate per GB of indexed data storage.
AWS CodeDeploy is free for code deployments to AWS EC2 instances, but there is a charge for each on-premises instance update. Any other AWS resources used during the code deployment, such as Amazon S3 buckets, will be charged accordingly.
The AWS AppStream application service is charged on an hourly basis for streaming resources actually used—plus a fixed monthly fee for each authorized user.
Pricing rates are tiered according to:
Another factor that impacts the AWS invoice is in which Availability Zone region(s) the service was provided—prices vary from region to region based on different latencies and data transfer speeds. Last but not least, if you have used AWS Organizations to apply centralized policies to multiple AWS accounts, the combined usage of all of the consolidated accounts is used to calculate volume discounts.
Note that AWS can also provide a Cost and Usage Report that is structured in the same way as the invoice and gives a near-real-time listing of estimated charges since the previous invoice.
The Cost and Usage Report is made available several times a day as a CSV file in a user-specified Amazon S3 bucket.
For each billable operation, the line item columns in the billing report provide important information on the type of usage, where it took place, when it took place, the resources consumed and the applicable rates, such as:
For each billable operation, the billing report provides detailed pricing information. This information includes:
These line item fields provide important information on Reserved Instances:
In addition to understanding AWS Storage Costs and Usage Reports and detailed monthly invoices, AWS provides tools to help its customers track real-time usage (e.g., the dashboard) or view historical usage and predict usage requirements (e.g., the Cost Explorer).
You can also assign key-value cost allocation tags to group your AWS costs according to, for example, your organization’s cost centers or applications.
Cost allocation tags can be particularly helpful in identifying underutilized resources, such as unused Reserved Instances -- visibility that you can use to optimize your AWS services.
Want to get started? Try out Cloud Volumes ONTAP today with a 30-day free trial.