Storage
Mobility
Protection
Analysis and control
TOPICShare
In the first article of this series, I reviewed the fundamentals of cloud resource costs using AWS as a reference. This showed how compute, storage, and network prices are affected by Availability Zone, performance SLAs, and type of tenancy. We also explained how savings can be achieved by pre-provisioning resources as opposed to on-demand usage.
Many organizations are aware that they do not manage cloud costs as well as they can. It will come as little surprise that, based on various industry and analyst reports across the board, IT owners believe at least 25% of their cloud spend is wasted, citing inefficiencies such as over-sizing of resources, inappropriate use of storage performance levels, and non-termination of idle or inactive workloads. The reality of the wastage is likely even higher.
Cloud cost optimization is one of the top initiatives of 2020. Companies of all sizes are responding to the challenge of optimizing cloud costs by establishing dedicated cloud teams as centers of excellence for architecting and managing cloud infrastructures.
In this post, I’ll look at some best practices that cloud teams can implement in order to contain cloud costs. I also introduce NetApp Cloud Insights, to present a unified view of cloud infrastructure usage and costs across hybrid and multi-cloud infrastructures. Armed with granular visibility, Cloud Insights can help companies reduce their cloud infrastructure costs by as much as 33%.
Best Practices
Although the previous post has tried to take the mystery out of AWS cloud resource costs, there’s no question that there are a lot of moving parts to track.
Governance and Ownership
Optimized cloud usage starts with clearly defined policies that encapsulate the organization’s cloud strategy and set out governance procedures and processes to ensure that the strategy is upheld across the organization. Although “governance” is often associated with large organizations, even the smallest company will benefit from explicitly establishing which workloads are suitable for the cloud (and which are not), who sets the target cloud budget, who is authorized to provision which resources, who oversees actual versus planned usage, and so on.
One of the most important principles of any cloud governance framework is who takes ownership of cloud spend. In most cases this ownership is distributed across different departments and lines of business, and tools must then be put in place to provide all stakeholders with the visibility they need to track and manage their cloud spend.
Cloud Provider Billing and Management Tools
All of the major cloud providers offer a suite of billing tools that provide ongoing visibility into the customer’s cloud usage and costs. Continuing to use AWS as a reference, the AWS Cost and Usage report is a CSV file updated three times per day and stored in a user-specified Amazon S3 bucket. The report aggregates line items on an hourly or daily basis, with each item providing details on a specific usage charge for a uniquely identified resource or service. At the end of the month, the report is finalized into a detailed monthly invoice.
In the case of pre-provisioned resources (Reserved Instances), the AWS Cost and Usage report shows both the effective costs incurred as well as reserved capacity that has not yet been utilized that month. In addition, the customer can consolidate billing across multiple AWS accounts into a single invoice. The master account pays for all charges aggregated by the linked member accounts, resulting in better leveraging of volume tiers as well as Reserved Instance discounts. However, each member account also gets its own bill so that they can track their usage and, where relevant, their chargebacks.
These billing tools provide a wealth of information, but the billing reports tend to be massive and difficult to parse for actionable cost-containment insights – I was certainly staggered the first time I attempted to decipher NetApp’s bill for our Hybrid Cloud lab! The providers themselves try to mitigate this challenge by offering advanced tools that make it easier to understand and manage resource utilization, such as the AWS Billing & Cost Management console.
Approved Resources
One of the key value propositions of cloud infrastructure is the frictionless ease with which resources can be provisioned in order to keep up with dynamic business requirements. Many organizations leverage this flexibility to ease development and production bottlenecks and accelerate business objectives. The other side of this self-provisioning coin, however, is that it can create an IT “Wild West” that not only wreaks havoc on budgets but can also undermine security requirements.
One way to find a middle ground between a cloud free-for-all and cumbersome centralized control is to understand your organization’s cloud use cases and support them with approved templates, service catalogs, and workflows that optimize cloud usage and spend. The trend towards Infrastructure As Code (IaC) is but one example of how an organization can standardize without forfeiting agility. There are several leading third-party IaC vendors such as Chef, Puppet, and Ansible. But the cloud providers also provide their own IaC services, such as AWS CloudFormation.
The Right Tooling Matters
Even if you implement these best practices and more, it can still be challenging to achieve the level of granular, real-time visibility required across your entire cloud infrastructure to facilitate rapid remediation of situations that unnecessarily drive up cloud costs—such as underused or unused compute or storage resources. It is in order to meet this challenge that NetApp has developed Cloud Insights, a managed service that provides actionable insights into cloud spend metrics across even the most complex hybrid and multi-cloud environments.
Cloud Insights collects resource usage data from virtually every major on-prem or cloud infrastructure platform and then presents a consistent non-vendor-specific view of the assets and their usage patterns. This provides actionable insights not just to specialized cloud, financial, or operations teams but to all stakeholders across an organization. Individual users, projects, departments and any other authorized party can access the dashboards and run intuitive point-and-click queries to get a clear understanding of how much infrastructure money is being spent (or wasted), where, and by whom.
Contrary to the AWS Cost and Usage report, and reports like it from other providers, there’s no need to start spreadsheet surfing. We’re building key use cases to be ready out of the box:
- Reduce over-provisioning: Collect cost metrics from all cloud infrastructures and attribute them to applications and owners. Because all metrics are part of a single and consistent data model, it is very easy to pinpoint over-provisioning wastage and trigger corrective actions across multiple providers.
- Spend analysis: Consistently attach costs to any type of resource to identify where wasted money is most prevalent in the organization, whether it’s in the cloud, or your own data centers.
Final Note
Sign up for a free trial of NetApp Cloud Insights and see for yourself how it unifies all of your cloud spend into a consistent, highly granular model that can significantly reduce cloud infrastructure costs.
