Kubernetes Is Being Reworked Around Idle, Long-Running AI Agents
Kubernetes was built for workloads that start, serve a request, and disappear. Agentic AI doesn't work that way. Agents hold state, sit idle while they wait on a model, an external service, or a person, and generate a growing stream of API calls. Here's the thing: the platform handles all of that poorly today, and the fixes are only starting to appear.
The idle pod problem
Agents spend a lot of time waiting. Right now each one typically gets its own pod, and that pod keeps its memory and CPU reserved through every pause. Multiply that across many agents and you're paying for capacity that does nothing (and this is the part most people overlook: the meter keeps running even when the agent isn't doing anything useful). Not a rounding error.
This is a mismatch of assumptions more than a bug. Kubernetes scheduling and resource requests were designed around workloads whose footprint is fairly predictable from one request to the next. A pod that sits idle looks, to the scheduler, almost the same as one that's busy. Similar systems that hold long-lived sessions tend to hit this pattern, where utilization graphs look healthier than the actual work getting done.
The control plane has a second issue. Persistent agent sessions push the number of objects it must track into the millions, a scale it was never designed to handle. Add more agent traffic on top and the control plane slows down right when demand is climbing. That's a bad time to slow down. The control plane is shared by everything in the cluster, so degradation caused by agent chatter doesn't stay confined to the agents.
Snapshotting agents instead of parking them
Google's response is Agent Substrate, an open source framework it contributed to the Cloud Native Computing Foundation (CNCF). It separates an agent's stateful session and identity from the compute running it. Idle agents are snapshotted to storage, and their workers go back into a shared pool. When an agent needs to resume, its state is restored in under 500 milliseconds on an available worker.
The idea will look familiar to anyone who's worked with checkpoint-and-restore tooling. What's new is applying it to agent sessions at the orchestration layer, where the platform decides when to pause and resume rather than the application doing it by hand. Whether 500 milliseconds is fast enough depends entirely on the workload. A conversational interface can probably absorb that delay, but a multi-step pipeline that resumes dozens of times per task will feel every one of them.
That's the part that matters for cost. You stop paying for memory and CPU that sit empty between requests, and the workers stay busy with other tasks. The economics only work, though, if pausing is cheap enough to be nearly free. That makes this a bet on storage performance and scheduling as much as on the framework itself. My skepticism is narrow but real: snapshotting is a sound pattern, and it's only as good as the state you actually capture. Agents holding open network connections or in-memory caches won't necessarily snapshot as cleanly as the diagram suggests.
Control plane performance
Mickey Boxell, senior principal product manager for cloud native services at Oracle, pointed to the control plane as the next pressure point. As agents make more calls to the Kubernetes APIs, and that volume climbs quickly, control plane performance becomes a serious concern. Boxell's framing moves the conversation from compute efficiency to API load. Those are different problems, and they tend to have different owners inside a platform team.
Alex Zenla, CTO at Edera, which makes a hypervisor optimized for running containers securely, is less worried. Kubernetes has been extended before, he argued. The community already moved stateful applications onto a platform designed for stateless workloads, so fixing deep architectural gaps is within reach. My read is that the answer will look more like new extension points than a rewrite of the core.
I'd be cautious about reading too much into that optimism. Extension mechanisms have carried a lot of weight in the Kubernetes ecosystem, but every new extension adds more objects, more watches, and more reconciliation work for the same control plane. That's exactly the pressure Boxell described. The extension mechanism can end up being part of the problem it was meant to solve, and that's the tension I'd keep an eye on.
Two different security problems
Securing agents is a separate track. Google has also contributed gVisor to the CNCF. It isolates code inside a lightweight userspace application kernel that can itself run as a container, which keeps untrusted code away from the host. Agents routinely execute code they generate, so that boundary matters more than it used to. Think of it as a wall around the code, not a filter on what the code is allowed to ask for.
NVIDIA this week contributed OpenShell, an open source runtime for deploying AI agents securely. The two tools solve different problems. gVisor limits what untrusted code can touch. OpenShell enforces policies on what an agent can access and what it can do. Don't treat them as substitutes. Expect to need both, one for the code and one for the agent's behavior.
Here's where teams tend to get tripped up. They hear "sandbox" and assume the agent is contained. A sandboxed process can still be steered, by a bad prompt or a poisoned tool response, into doing something harmful within the access it already has. If you're working in this space, the policy layer is the part you'll have to define yourself, because your access rules depend on your data, your systems, and your risk tolerance. No upstream project can write those for you.
Timeline and what to plan for
Don't expect any of this in your distribution soon. Some of these efforts could take a year to show up in mainstream Kubernetes distributions, and nobody has pinned down when they'll reach the clusters IT teams run in their own data centers. That gap between upstream and production is where most of the pain lives. Projects land in open source foundations, get picked up by vendors, get tested, and then get packaged, and each step adds time.
Some teams are already evaluating alternatives. The source doesn't name them, so the practical question is what your current vendor supports and what it plans to support. Until these pieces reach your distribution, you'll either build workarounds or accept the current costs. Neither is free, and the workarounds tend to become permanent if nobody schedules time to retire them.
The direction is still clear. The Kubernetes that eventually runs agentic workloads will look quite different from today's clusters, with agent state separated from compute and a control plane sized for agent-scale traffic. If you're planning for it, start by measuring how much of your cluster sits idle. That's where the first savings and the first strain will show up. I'd add a second measurement: API request volume per workload, not just CPU and memory. The control plane pressure Boxell and Zenla describe won't show up in node utilization charts until it's already hurting.
What this signals for Kubernetes
This is more significant than it looks. Kubernetes spent years becoming the default platform for stateless services, and agents are asking it to treat state, long idle periods, and bursty API traffic as first-class concerns. Those were never the design center. Every fix in this space is, underneath, a decision about what the platform is for.
The fact that Google, NVIDIA, and vendors such as Oracle are all working on pieces of this tells you the pressure is real across the ecosystem. It doesn't tell you which approach wins. Agent Substrate, gVisor, and OpenShell are different bets on different layers, and the ones that stick will probably be the ones that fit existing Kubernetes primitives rather than replacing them. Projects that ask operators to learn an entirely new model tend to struggle with adoption, however good the underlying idea is.
And yet the risk is familiar. Platform layers that promise to hide state management usually move the complexity somewhere else. It shows up in debugging snapshots that don't restore the way you expected, in policies that conflict with each other, and in storage tuning nobody budgeted for. That's the trade I'd watch most closely, more than any single performance claim. Agents may force Kubernetes to change, but the people running it will still be the ones absorbing the surprises.
