Shifting Focus: H3C Advocates for Token Efficiency in AI Infrastructure

In the rapidly evolving field of AI infrastructure, the earlier focus on simply scaling compute power is giving way to a more nuanced question: How effectively can existing computational resources be utilized? This shift emphasized by H3C during the 2026 Apsara Conference, highlights the need for systems that enhance Token efficiency as AI agents increasingly rely on expansive foundation models.

Adding GPUs to the mix does not guarantee a proportional increase in performance. Factors such as idle compute resources, network congestion, and the complexities involved in managing large clusters can diminish the anticipated benefits of expanded processing capacity. H3C's showcasing of its product suite at the conference, thus, isn’t just about offering more hardware; it presents a clear strategy that prioritizes system-level optimization across the entire AI infrastructure.

Efficiency Over Quantity

At the heart of this evolution lies a fundamental rethinking of what constitutes performant AI clusters. Zhu Shiyin, general manager of H3C’s Advanced Technology Research Department, remarked that AI clusters should be seen as tightly integrated systems. Simply stacking GPUs isn’t sufficient; the coordination between hardware and software becomes imperative to realize the full potential of these systems. As clusters scale from hundreds to thousands of GPUs, the entire ecosystem—ranging from chip-to-chip communication and data transfer to task scheduling and power supply—plays a crucial role in effective compute utilization.

An illustration of this philosophy is the UniPoD S80000 Series SuperPod, presented by H3C at the event. With configurations supporting between 32 and 1,024 GPUs and scaling up to 16,384 GPUs, this system is also designed for heterogeneous computing, including CPUs, GPUs, NPUs, and DPUs. The emphasis here is clear: the competition in AI infrastructure is transitioning from sheer GPU count to the efficiency of Token output from each GPU.

Network Bottlenecks

Another significant challenge identified by H3C is the potential for network bottlenecks. As GPU performance accelerates, so does the volume of data that needs to be transferred among them. If networking cannot keep pace, it risks halting the processing momentum of GPUs. This has prompted H3C to demonstrate three distinct interconnect scenarios: Scale-Up, Scale-Out, and Scale-Across, addressing the various dimensions of network performance.

The introduction of high-speed interconnects is critical. For instance, the S9828-128EO switch employs silicon-photonics to cut latency by 15%. Meanwhile, the S9828-64FP switch uses advanced technologies to save over 20% in power consumption while supporting expansive clusters. At a broader level, the 800G DCI switch S12500R-64EP aims to support compute scheduling and synchronization across data centers. The takeaway here is that interconnects must evolve beyond mere bandwidth concerns, factoring in reliable protocols, congestion management, and overall resilience to avoid becoming performance limiting factors.

Data Management Challenges

Moreover, the significance of data in AI operations cannot be overlooked. Effective GPU operation requires substantial data movement during model training, parameter exchange, and inference processes. Given this context, H3C has integrated high-performance storage solutions into its optimization narrative. The UniStor X20000 series, for example, can offer remarkable bandwidth capabilities along with high input/output operations per second (IOPS), crucial factors for efficient data handling.

Notably, these advancements can minimize GPU waiting times and expedite inference scenarios, making high-performance storage an integral player in the Token production pipeline. This evolution indicates a broader trend where AI infrastructures are transitioning from mere computing engines to sophisticated data processing systems, recognizing that delays in any phase can translate into significant cost implications.

The Software Component

As the hardware side reaches maturity, the importance of software optimization becomes increasingly pronounced. Zhu points out that the collaboration between operating systems, communication libraries, resource management tools, and hardware can streamline operations significantly. Effective communication strategies and dynamic task management can reduce idle times and improve resource utilization dramatically.

These dynamics suggest that modern AI infrastructures are morphing into complex super-systems where chips, networks, storage units, and cooling systems must cooperate synergistically. The software layer orchestrates these elements, ensuring that all parts function together efficiently. Should any facet of this integrated system falter, it could lead to ripple effects negatively impacting the overall productivity of the AI system.

Preparing for the Future of AI Infrastructure

At the Apsara Conference, H3C repeatedly underscored a critical emerging theme: the competition landscape for AI infrastructure is shifting. It's not merely about amassing more GPUs but about optimizing each resource for efficiency. As we move towards deploying AI at scale—with models increasingly climbing toward trillion-parametric structures—parameters like compute utilization, Token throughput, latency, energy consumption, and cost-effectiveness will dictate leadership in the space.

This perspective illustrates that the focus on extreme efficiency in Token production is not just about hardware aggregation but rather a pervasive mentality aimed at maximizing every component of the infrastructure. As AI embraces wide-scale production, compute challenges evolve beyond mere chip capabilities, signaling a complex race focused on holistic system efficiency.