Efficient Multi-Tenant Cache Solutions in Kubernetes with Dragonfly and vCluster
A reproducible lab utilizing Dragonfly and vCluster reveals how a central caching layer for model distribution can effectively share large AI model files among different tenant workloads while preserving their security boundaries.
When you think about multi-tenant Kubernetes environments, every team having its own API boundary may seem ideal for independence, but it inevitably leads to redundant downloading of the same sizable AI model files. This repetition results in unnecessary network bandwidth consumption and duplication of stored data.
A cache dedicated to each tenant can ensure they have access to necessary data, but it also restricts opportunities for data sharing. On the other hand, a node-level cache allows for broader data accessibility but comes with the risk of needing elevated permissions and network access, which could compromise security. My approach involved setting up a small testing lab to investigate whether a shared, platform-owned peer-to-peer (P2P) cache across virtual clusters can enable efficient reuse of the same model payload across different tenants while keeping their respective workloads isolated.
Understanding the Architecture
In this experiment, Dragonfly functioned on the host cluster without a manager. Key roles were played by schedulers and seed peers to coordinate tasks, while a client DaemonSet operated on each node to facilitate model downloads. The client also set up an HTTP proxy on port 4001 using host networking, ensuring that the platform handled the overall system and the cache.
Each tenant—referred to as Tenant A and Tenant B—was configured using separate open-source APIs via vCluster. Their model download tasks were executed as standard, non-root Pods, devoid of any special privileges. The Jobs utilized the Kubernetes downward API to identify their node IP and sent model payloads through the local Dragonfly proxy.
The download process began with a HEAD request sent to Hugging Face, resolving the signed CDN URL for the required model file. The subsequent download through Dragonfly was streamlined by a proxy setup that eliminated specific signed query parameters from the cache identity. This design was crucial as a changing signed URL could block effective reuse between tenants.
Figure 1. Host-managed Dragonfly data plane shared by two vCluster tenants.
Results from the Experiment
During testing, Tenant A operated on the host control-plane node and downloaded a model payload of 267,954,768 bytes, with a resulting SHA-256 checksum of 5e3f1108e3cb34ee048634875d8482665b65ac713291a7e32396fb18f6ff0063, completing the task in approximately 11 seconds.
Next, I placed Tenant B on a different node (dragonfly-host-worker2) to ensure that its request could not utilize Tenant A's local cache. Despite this, Dragonfly ensured that both requests shared the same task ID. As Tenant B processed its request, the scheduler pointed to the control-plane client and a seed peer for data retrieval. The logs confirmed that the control-plane client transmitted existing data pieces to Worker 2. Ultimately, Tenant B received the identical 267,954,768-byte model file and also computed the same checksum, with completion taking about three seconds.
What’s noteworthy here is that the shorter duration isn’t the only evidence of success; the shared task ID, parent list, and logs confirming remote data delivery constitute a reliable basis for demonstrating inter-tenant model sharing. While total origin bytes weren’t directly measured, it’s clear this approach significantly reduces unnecessary repeated downloads.
Key Takeaways for Platform Engineering
There are several important takeaways from this experiment. First, treating the cache as a dedicated platform service allows tenants to benefit from data access through a narrow interface while the platform itself handles deployment, scalability, and ongoing maintenance.
Second, managing cache identity is a critical architectural consideration. Careful normalization of signed model-hub URLs was essential in this experiment, and similar strategies must be approached cautiously when dealing with private or gated models, ensuring proper authorization and data isolation.
Lastly, logs detailing transfer sources carry more significance than response time metrics. A quicker second download could indicate reuse but does not clarify if the data originated from a local cache, a remote peer, or the actual source. This study did not delve into adversarial tenant interactions but rather tested standard workloads with strict access controls.
Other Options and Production Best Practices
Beyond the methods tested with Dragonfly and vCluster, options such as per-node registry mirrors, pull-through caches, or shared persistent-volume caches could also be effective. The decision on which method to employ is contingent on factors like artifact types, security boundaries, and administrative responsibilities.
Moving forward, a solid production guideline emerges: consolidate the distribution of repeatedly used artifacts within the platform’s capabilities, offer a minimal tenant interface, and diligently verify the source of transfers. Starting with public artifacts and explicit cache keys, while treating private model management as a distinct requirement, is fundamental for success.
Author Bio
Pavan Madduri serves as a Senior Cloud Platform Engineer at W.W. Grainger and holds the role of Tech Lead for the CNCF’s TAG Workloads Foundation through 2028. His contributions to Dragonfly, a prominent CNCF project, alongside being a vCluster Ambassador and certified CNCF Golden Kubestronaut, illustrate his commitment to advancing cloud-native technologies.
Disclosure: The author is part of the vCluster Ambassador Program. This article solely employs open-source vCluster features and has not been reviewed or sponsored by vCluster Labs.
