GKE Cost Optimization: Rightsizing Workloads, GPUs and Spot Capacity
Google’s Kubernetes experience gives Google Kubernetes Engine (GKE) powerful tools for managing infrastructure. Yet a 2026 report puts average GKE CPU utilization at 8%, memory utilization at 20% and GPU utilization at 6%. The gap points to a practical limit: a managed platform can provision and schedule resources, but it can’t know how much compute an application actually needs.
The 2026 State of Kubernetes Optimization Report also found that GKE clusters overprovisioned CPU by 69% and memory by 79%. Its findings draw on direct measurements from more than 23,000 Kubernetes clusters and millions of compute resources across AWS, GCP and Azure, collected before automated optimization was enabled.
My read is that GKE’s utilization problem is less about missing infrastructure features than about the decisions encoded in workload requests. Google created Kubernetes and has operated it at enormous scale, experience reflected in GKE features such as Autopilot. But when a workload declares that it needs four CPUs, the platform has little basis to decide that the application can run comfortably on fewer.
Start with workload requests
Resource requests are a useful first place to investigate because oversized values affect both cost and capacity planning. In Autopilot, an inflated request can directly raise the cost of running a workload. In Standard, it can also influence scheduling and lead the cluster to provision more node capacity than the application consumes.
There’s a meaningful difference between allowing headroom and carrying outdated padding. Production workloads need room for real changes in traffic and demand. But a value chosen cautiously can become permanent long after the conditions behind it have changed.
Compare declared CPU and memory requests with actual usage over a representative period. For services with weekly traffic patterns, seven to 14 days can reveal more than a quiet-hour snapshot. The aim is to understand normal behavior and its variation before changing requests, so that a lower figure still accounts for production needs.
Consider a service that requests two CPUs and 2 GiB of memory but typically uses about 500 millicores and 700 MiB. Adjusting the node pool won’t resolve the mismatch: Kubernetes still sees a workload asking for much more than it usually consumes. A controlled trial across a small set of workloads can show whether revised requests hold up before teams extend the change more broadly.
Coordinate autoscaling with rightsizing
Automated tuning needs care when Vertical Pod Autoscaler (VPA) and Horizontal Pod Autoscaler (HPA) operate on the same signals. VPA can revise resource requests, while HPA can use CPU or memory utilization to change the number of pods. If both respond to one another, VPA changes can alter HPA calculations; the resulting pod count can then change utilization and feed back into VPA.
Starting VPA in recommendation-only mode lets teams inspect proposed changes before applying them. When both autoscalers are needed, application-level HPA signals such as request rate or queue depth can avoid tying horizontal scaling directly to the resource values VPA is adjusting.
The report suggests rightsizing needn’t mean accepting worse reliability: automated rightsizing cut provisioned CPU by roughly half while reducing out-of-memory kills to near zero. That result doesn’t mean every service should receive smaller requests. It shows the value of using production data to distinguish useful headroom from capacity that rarely serves the application.
Give GKE Standard accurate inputs
After pod requests are better aligned with observed use, teams running GKE Standard can make more informed choices about the underlying Compute Engine fleet. Cluster Autoscaler remains the main mechanism for adjusting node capacity, while Node Auto Provisioning can create node pools when workloads need machine types not already available in the cluster.
Autoscaling can respond to declared demand; it can’t tell whether that demand is inflated. If ten services each request twice the CPU they normally consume, the autoscaler has reason to provision capacity for those requests. More accurate requests give autoscaling and node consolidation a better basis for placing workloads efficiently, choosing suitable machine types and removing capacity that’s no longer needed.
That control comes with responsibility. Standard gives operators more say over the nodes, along with ownership of the idle capacity those nodes carry. In Autopilot, teams take a different path: workload requests have a direct role in what they pay, so rightsizing remains central even when infrastructure management is more abstracted.
Use Spot for workloads that can pause
Once teams understand their baseline capacity needs, they can decide which workloads require on-demand availability and which can tolerate interruption. GKE Spot VMs offer lower-cost capacity, but Google’s model provides roughly 30 seconds of preemption notice. That makes Spot a poor fit for workloads that cannot handle a sudden loss of a node.
Batch jobs, CI/CD runners, stateless services with enough replicas, and machine-learning training jobs that checkpoint their state are more natural candidates. A sound setup also accounts for disruption budgets, termination handling and an on-demand fallback if Spot capacity disappears. Without those safeguards, a compute saving can turn into an availability issue.
Spot works best as one part of a capacity plan. Keep workloads that need predictable availability on suitable reliable capacity, and allow interruption-tolerant work to use Spot when it’s available.
Improve the return on GPU and CPU capacity
Some inefficiency calls for different hardware choices rather than smaller requests. The report found average GKE GPU utilization of 6%, with substantial variation during the day, and said GPU sharing remained relatively uncommon. When inference needs only part of a GPU much of the time, assigning an entire device to one workload can leave expensive capacity idle. Sharing or partitioning may help multiple workloads make better use of the same hardware, depending on the GPU and application.
One production example in the report combined GPU time-slicing for seven models with improved bin-packing and Spot usage. The organization reduced costs by more than 70% compared with its previous platform while maintaining latency. The useful measure is the work the hardware completes, not utilization as a number in isolation.
CPU architecture offers another option for workloads that can run on it. ARM accounts for about 9% of the Kubernetes CPU fleet and has grown 3.5 times faster than x86 since Q2 2024, according to the report. GKE supports ARM through T2A and Google’s Axion-based C4A and N4A families.
A focused migration can establish whether ARM suits a particular service: move a small group of scale-out workloads that support linux/arm64, then compare performance and cost. Changing processor type won’t correct an oversized request, though. Rightsize first so the new hardware isn’t simply carrying the same mismatch at a lower price.
Commit only after the baseline is clear
Pricing commitments make more sense once teams know which capacity they consistently need. GCP’s Sustained Use Discounts apply automatically to eligible Standard node usage; Committed Use Discounts require a deliberate commitment against a baseline. Committing before measuring actual demand can lock in today’s excess capacity for years.
A practical sequence is to align workload requests with production behavior, improve node placement, assign interruption-tolerant jobs to Spot, and test GPU sharing or ARM where workloads fit. Then teams can estimate the stable baseline separately from burst capacity and decide how to price the capacity they expect to keep using.
GKE’s automation handles many infrastructure tasks that become difficult at scale, but practitioners still shape efficiency through workload requests and capacity choices. Small, measured changes provide a way to improve utilization while observing how services behave in production. The next useful optimization is usually the one backed by workload data, not a fleet-wide adjustment made on assumption.
