Overprovisioning of Containerized Applications Adds to Cloud Costs 

Organizations are struggling with cloud cost management for Kubernetes, with the inability to scale back once application demand drops as one of the major drivers of wasted cloud spending.

This was one of the chief findings of a CAST AI report providing insight into overprovisioning of resources, overspending on cloud costs and the resulting waste in energy consumption.

The study found, on average, that more than a third (37%) of CPUs for cloud-native applications are never used and by adding spot instances to applicable workloads, organizations could save up to 60% in dollar terms.

Clusters were selected for analysis based on the highest activity level running applications on the three most popular cloud services globally: Amazon Web Services (AWS), Microsoft Azure and Google Cloud Platform (GCP).

Laurent Gil, cofounder and CPO of CAST AI, explains there are a few reasons for overprovisioning, including the fact that it is an “impossible task” for a human.

“Overprovisioning is a modus operandi for any DevOps team,” he says. “Think about it: Developers and DevOps pros would have to select the right mix of VMs among 600 different shapes, memory and processors. And they’d have to do that every few seconds.”

Add to this that the main priority of DevOps teams has always been uptime, and you get vast overprovisioning built-in for almost all DevOps teams, Gil says. 

He explains some best practices for cloud cost management for Kubernetes include tracking cost-per-provisioned-CPU versus requested CPU.

“Why differentiate between the two? If you compare the number of requested versus provisioned CPUs, you can find a gap,” he says. “This gap is your cloud waste.”

If you’re running a Kubernetes cluster that has yet to be optimized, you’ll see a difference between how much you’re provisioning and how much you’re actually requesting.

“You spend money on provisioned CPUs and only end up requesting a small amount of them, so the price of individual requested CPUs grows,” Gil says. “Calculate how much you’re spending per requested CPU to make cost reporting more accurate.”

He adds it’s also important to choose the right type and size of virtual machines, which starts with defining the organization’s requirements to provision only as much capacity as the workload really needs.

“Cloud providers offer many different instance types matching a wide range of use cases with different combinations of CPU, memory, storage and networking capacity,” Gil says. “Each virtual machine type comes in one or more sizes to help you scale easily.”

A third tip is to optimize Kubernetes autoscaling and ensure autoscaling policies don’t clash. 

“VPA automatically adjusts the requests and limits configuration to help you reduce overhead and cut costs. HPA, on the other hand, scales out – and more likely up than down,” he says. “That’s why you should make sure that your VPA and HPA policies aren’t interfering with each other.”

He adds that when designing clusters for business- or purpose-class tier of service, it’s a good idea to review binning and packing density settings.

Gil points out that spot instances are another great way to cut your Kubernetes bill, as they offer discounts reaching even 90% off the on-demand pricing.

“But before jumping on this opportunity, take a look at your workload to see if it’s a good fit for spot instances,” he advises. 

He points out spot instances offer an incredible cost-saving opportunity, but they’re not a silver bullet and they might not be a good fit for every application out there.

“That’s why it’s important to combine spot instance usage and automation with other cost management techniques,” he says. “For example, automated right-sizing, where the solution chooses the most cost-efficient virtual machine type and size for a given workload.”

Finally, Gil recommends using an automation tool that does cloud optimization for the business. 

“AWS alone has some 400+ virtual machines on offer,” he notes. “What if your teams use different cloud providers? The manual effort of configuring resources, picking virtual machines and setting autoscaling policies will cost you more than its optimization impact.”

He points out the market is full of cloud cost optimization and management solutions that take some or all of the above tasks off engineers’ shoulders, reclaiming time for teams to do more strategic work. 

Nathan Eddy

Nathan Eddy is a Berlin-based filmmaker and freelance journalist specializing in enterprise IT and security issues, health care IT and architecture.

Nathan Eddy has 5 posts and counting. See all posts by Nathan Eddy