EKS Cost Optimization: Rightsizing, GPU Sharing and Smarter Spot Use

Amazon EKS gives platform teams a wide range of compute choices, but that breadth hasn’t translated into efficient use of the capacity they buy. The 2026 State of Kubernetes Optimization Report, based on production clusters measured before automated optimization, shows average CPU utilization across AWS, Azure and Google Cloud falling from 10% to 8% in 2025. Memory utilization slipped from 23% to 20%, while CPU overprovisioning rose to 69% and memory overprovisioning reached 79%.

For EKS operators, the signal is clear: adding instance types and autoscaling options doesn’t resolve waste when workloads ask Kubernetes for far more capacity than they consume. The same gap is especially costly for GPUs, where average utilization was 5% across the report and on AWS. The practical challenge is to match capacity to real demand without compromising availability.

The 2026 State of Kubernetes Optimization Report draws on tens of thousands of clusters and millions of compute resources. Its results point to several operational levers: tune workload requests, share accelerators, place flexible workloads where Spot capacity makes sense, and rightsize before committing to long term discounts.

Autoscaling can faithfully reproduce bad estimates

Karpenter can provision EC2 capacity quickly in response to pending pods, without requiring teams to maintain fixed node groups. But it schedules against Kubernetes resource requests. If a pod requests 4 vCPUs and 8 GiB of memory, Karpenter must find room for those amounts even if the application uses only 500 millicores and 1.5 GiB at peak.

That distinction helps explain why a sophisticated node autoscaler can coexist with low utilization. Autoscaling responds to declared demand; it doesn’t determine whether those declarations reflect actual consumption. Teams often inflate requests to guard against CPU throttling, latency spikes or out of memory evictions. Because oversized requests rarely trigger an immediate incident, they can persist in Helm charts and Terraform templates long after the original concern has passed.

On EKS, teams running their own EC2 nodes pay for the spare capacity that those requests reserve. The cluster may be correctly scheduled according to its configuration while the configuration itself remains detached from application behavior. A useful optimization loop therefore has to examine workload consumption as well as node demand.

GPU sharing changes the economics of inference

Idle GPU capacity carries a steep cost, and the pressure is rising. The report puts average GPU utilization at 5%. In January 2026, AWS raised H200 Capacity Block prices by 15%, citing supply and demand imbalances. Yet variable inference traffic can leave dedicated accelerators waiting between bursts, while customers continue paying for the full instance.

For suitable workloads, time slicing and bin packing offer a way to place several models on shared physical GPUs. ALLEN Digital, an edtech platform, ran seven machine learning models on AWS EKS, including BGE-M3, LlamaGuard and Multilingual E5 Large. It moved from dedicated Amazon SageMaker instances to EKS, then combined GPU time slicing, automated bin packing and a 50/50 mix of Spot and on demand capacity. The company reported compute cost reductions of more than 70% against SageMaker, with no compromise in inference latency.

That example illustrates a practical distinction: provisioning more accelerators and using existing ones more effectively are different strategies. For workloads with intermittent demand, sharing can increase useful work per GPU before an organization seeks additional capacity.

Spot savings depend on where workloads run

EC2 Spot Instances can cost up to 90% less than on demand capacity, but fewer than 2% of GPUs in the report were using Spot. The risk of interruption, which comes with a two minute warning, makes teams cautious about putting production workloads there. The report’s regional data suggests that this decision needn’t be uniform across a fleet.

For NVIDIA T4 instances of type g4dn.xlarge, Spot survival probability in eu-west-3 stayed above 90% over a 24 hour window. In us-east-1 and eu-central-1, it fell below 20% within that period. Regional pricing also varied: Spot rates for p4d.24xlarge A100 instances differed by roughly two to nearly five times among us-east-1, us-east-2 and us-west-2. The report says shifting workloads in response to market conditions delivered savings of up to 80% against average Spot prices.

Those patterns make manual Spot management difficult at scale: prices and interruption risk can change by region and availability zone. Iterable, a marketing platform, used automated hourly Spot rebalancing across its AWS footprint. The company eliminated 2,000 unneeded CPUs, reduced its node count by 10% and cut compute spend by 75% while handling changes in real time traffic.

Graviton adoption is growing with stateless workloads

ARM based nodes are gaining ground in Kubernetes fleets. According to the report, they’ve grown 3.5 times faster than x86 nodes since the second quarter of 2024 and now account for 9% of total Kubernetes CPU capacity. The finding aligns with the report’s observation that 93% of deployments are stateless.

Stateless APIs, microservices and background workers are often good candidates for Linux on arm64, since they don’t carry the same persistent volume constraints as stateful services. Moving an oversized workload to Graviton can lower its per core cost; reducing its resource requests first can also eliminate capacity the application doesn’t need. Combining those steps can address both allocation and instance cost.

Static buffers don’t guarantee reliability

Platform teams often keep broad resource headroom to protect service level objectives. But overprovisioning doesn’t automatically protect a workload that has outgrown its limits or developed a memory leak. In one production environment examined by the report, out of memory kills averaged 40 to 50 per measurement window and sometimes exceeded 80, despite extensive resource padding.

Static buffers can leave quiet services with excess capacity while a fast growing service remains constrained by an outdated limit. Automated rightsizing can reduce requests for workloads that consistently use less than they reserve, while raising limits for applications showing memory pressure. The report describes one environment where this approach brought out of memory kills close to zero and reduced provisioned CPU by about 50%.

Commit to the baseline after removing waste

Savings Plans and Compute Reserved Instances can reduce cloud costs, but buying commitments before tuning workloads risks discounting capacity that applications don’t use. The report says manually managed workloads and shifting traffic can leave organizations wasting up to 35% of prepaid commitments.

A more measured sequence is to rightsize requests using production consumption data, pack nodes more efficiently, and direct interruption tolerant workloads to Spot with multi availability zone placement and on demand fallback. Then commitments can be based on the remaining baseline; the report identifies commitment utilization near 98% as a target. Teams can find more detail in this guide to EKS cost optimization.

The same utilization problem appears in AKS and GKE, suggesting it stems from operating Kubernetes at scale rather than from one cloud’s instance catalog. With more teams, regions and workload types, resource requests and node choices drift faster than manual reviews can keep up. For EKS operators, the durable approach is to keep measuring actual demand and feed those observations into workload sizing, placement and purchasing decisions.