Author | Wang Han Editor | Mo Ying
Since the viral success of "Crayfish" OpenClaw at the beginning of the year, the explosion of AI agents has become unstoppable. From phased training and inference of single models to large model training, multi-model collaborative inference, and the continuous operation of massive agents... AI tasks are entering a new "always-on" stage. At the World Artificial Intelligence Conference 2026 (WAIC 2026), Zheng Weimin, an academician of the Chinese Academy of Engineering and a professor in the Department of Computer Science and Technology at Tsinghua University, pointed out that the token consumption of a single agent can reach 100 to 1,000 times that of traditional conversational applications.
As massive agents concurrently invoke models, read and write memory, orchestrate tasks, and execute tools, the token consumption rate is surging at an exponential rate. The skyrocketing demand for tokens has directly raised the scale ceiling of AI infrastructure. Thousand-card clusters are no longer sufficient, and ten-thousand-card and hundred-thousand-card clusters are becoming the standard for cutting-edge large model training.
However, power and space in data centers are limited. To meet the boundless demand for tokens, the core direction is to pack more computing power into a unit space, pushing the computing power density to the extreme. As computing power density increases, the power consumption per rack will inevitably surge, leaping from tens of kilowatts to hundreds of kilowatts, and further advancing toward the MW level. But here the problem arises. When the power per rack reaches this magnitude, traditional cold plate liquid cooling solutions will lead to inadequate rack cooling, insufficient space, and disrupted interconnects. These three bottlenecks are narrowing simultaneously within the limited space.
For token factories to continue expanding production, computing, cooling, interconnects, and power supply can no longer operate in silos; new solutions must be found from the system foundation. Native liquid cooling, a solution that couples computing, cooling, interconnects, and power supply at the system level, is beginning to emerge in the industry's field of vision.
01.The Surge in Token Demand Exposes the Triple Dilemma of Traditional Liquid Cooling
The explosion in token demand has driven up the power per rack and pushed deployment space to the limit. In such scenarios, the shortcomings of traditional liquid cooling are drastically amplified. Solutions that originally functioned well under low power density are now flashing red lights simultaneously across three critical dimensions.
The first dilemma is that computing power density is constrained by whole-system thermal management. High computing power density means integrating more chips and even whole-system components such as HBM, memory, NICs, and optical modules within a unit space, which also implies the need to dissipate more heat. Under high-density deployment, the traditional cold plate liquid cooling approach of "adding plates one by one" will ultimately fail to keep pace with the growth rate of whole-system heat generation. Once the cooling capacity falls behind, chips will hit the thermal wall and throttle down, causing token output to continuously decline from its peak. If cooling cannot keep up, token generation slows down.
The second dilemma is that cooling structures continuously encroach upon computing space. Currently, cold plate liquid cooling accounts for 93.5% of the liquid-cooled data center market, but the underlying logic of the vast majority of solutions remains stacking fans, cold plates, hoses, and fluid distributors layer by layer. The layered stacking of cooling structures constantly squeezes the space originally belonging to computing units, further compressing the maximum GPU loading capacity per rack. As rack power advances toward the MW level, this spatial contradiction is drastically amplified: the more complex the cooling, the more space it encroaches upon, and the fewer computing units can be accommodated. Thus, the maximum token production capacity per rack is locked.
The third dilemma is that the conflict between cooling and interconnects is amplified at the MW level. The expansion of the scale-up computing domain requires more computing nodes and switching nodes to form a unified system with high bandwidth and low latency. However, in traditional architectures, backplanes, cables, air ducts, cold plates, and liquid paths are planned separately, which lengthens interconnect paths, increases the number of interfaces, and raises communication latency. When dozens or even hundreds of GPUs need to synchronize data frequently, even minor latency will be multiplied, systematically depressing the overall efficiency of multi-card parallel inference. If interconnect efficiency is compromised, token generation slows down.
The triple dilemma points to the same conclusion: The problems of token factories have gone beyond "cooling" itself, becoming a systemic predicament where computing, interconnects, power supply, and cooling mutually constrain each other within limited space. To simultaneously break free from these three dilemmas, it is imperative to rethink from the very source of system design—planning computing, cooling, interconnects, and power supply as a unified whole.
02.Native Liquid Cooling: Addressing the Triple Dilemma from the System Source
So what is the right approach to "rethinking from the source"? Making the cold plate a bit larger, replacing hoses with thicker ones... these conventional ideas can only treat the symptoms. To cure the root cause, we need a paradigm shift—native liquid cooling. Traditional liquid cooling follows the logic of "defining computing first, then configuring cooling," finding space to cram liquid cooling equipment after the computing system is finalized. Native liquid cooling is not the "Pro Max" version of cold plate liquid cooling; its essential differences are reflected in three dimensions: in the time dimension, thermal design shifts from post-positioned to pre-positioned, so no one has to compromise; in the spatial dimension, cooling evolves from fragmented coverage to holistic management, no longer fragmented; in the logical dimension, the optimization target elevates from a single cooling component to the entire computing system, where computing, cooling, power supply, and interconnects no longer operate in silos. This systematic reconstruction unfolds precisely along computing power density, spatial density, and interconnect density, ultimately all pointing to token production capacity.
Let us first look at computing power density. Native liquid cooling adopts a whole-system thermal design that synergizes cooling and computing architectures. Through integrated cold plates and holistic liquid cooling reconstruction, it achieves collaborative cooling for the entire computing system. It breaks away from the cognitive inertia of modular local optimization in traditional cold plate liquid cooling architectures, bringing GPUs, CPUs, memory, NICs, optical modules, and SSDs all into unified thermal management. The computing power density of the entire rack can be increased to more than 10 times that of traditional solutions.
With improved cooling capacity, chips do not throttle down under sustained high loads, keeping token output stable at the peak without decaying over time. Next, let us look at spatial density. Through system structure reconstruction, native liquid cooling fully releases computing deployment space. Adopting integrated cold plates and minimalist engineering design, it achieves zero hoses, zero cables, and zero fans, reducing assembly interfaces by 80% and flow resistance by 60%, while freeing up 50% of equipment space for core units such as chips, memory, and interconnects. Within the same rack space, the number of computing units that can be accommodated leaps significantly. Cooling no longer encroaches on space, and the maximum token production capacity per rack is thoroughly unlocked.
The breakthrough in interconnect density is equally important. Native liquid cooling adopts a midplane-less orthogonal direct-connect architecture, decoupling computing nodes from switching nodes, and building a high-density, low-latency direct-connect network matrix through the shortest front-to-back paths. Based on the high-speed direct-connect design of 4.8TB/s between a single GPU and the switch, the 64-card fully interconnected architecture can provide 300TB/s of interconnect bandwidth, supporting 224G high-speed data transmission. Shorter interconnect paths mean lower communication latency and higher multi-card collaboration efficiency, enabling more computing resources to work together efficiently, systematically elevating the overall token throughput capacity of the cluster.
03.The Agent Era Requires Two Types of Computing Power Foundations: Inspur Information Covers Both with Native Liquid Cooling
From the triple dilemma to breakthroughs in three dimensions, native liquid cooling has proven the value of systematic reconstruction. But after the architectural logic is established, the next more practical question is: How do these capabilities ultimately materialize into products? What kind of hardware support does the computing power foundation in the Agent era actually need? To answer this question, we must first understand how agents work.
From receiving instructions to completing tasks, an agent must go through multiple steps such as tool invocation, memory read/write, task orchestration, and parallel scheduling. These "orchestration and scheduling" tasks primarily run on CPUs. In other words, the complete workflow of an agent "walks on two legs": the GPU is responsible for high-speed token generation, while the CPU handles efficient agent scheduling. Both types of computing power are indispensable; if either end fails, the response speed of the entire agent system will be dragged down.
IT infrastructure products, solutions, and services provider Inspur Information has launched corresponding high-density whole-rack products based on the native liquid cooling architecture to address these two types of demands. Looking first at the GPU side, Inspur Information's GPU native liquid-cooled whole-rack server can accommodate thousand-card computing power within a 48U space—the computing power density per rack reaches more than 10 times that of traditional solutions. It innovatively adopts an integrated SDAC cold plate, deploying heat sources such as GPUs, CPUs, memory, NICs, and optical modules in a high-density, two-sided planar layout across the entire cold plate, doubling cooling efficiency. In terms of interconnects, it pioneers a midplane-less MDP matrix direct-connect architecture, where 1,024 GPUs and switches form a high-density fully interconnected matrix through high-speed direct connections, supporting ultra-large-scale lossless expansion of thousands and tens of thousands of cards.
The final report card shows: single-rack power covers 330kW to 1MW, and large model throughput reaches 1.46 million tokens/s. A single MW-level rack can support a high-yield token production line, outputting at full speed continuously without throttling or slowing down. But having GPUs alone is not enough. Inspur Information simultaneously launched the CPU native liquid-cooled whole-rack server. Based on the liquid-cooled OCM architecture, a single rack integrates 384 CPUs, supporting the high-concurrency operation of over 40,000 agents, ensuring no frequency drops or speed reductions under high loads, keeping agent scheduling responses consistently in a low-latency state.
The GPU whole-rack server is responsible for high-speed token production, while the CPU whole-rack server handles efficient agent scheduling—the two types of computing power foundations work collaboratively under a unified native liquid cooling architecture. From token generation to agent execution, the entire link no longer has cooling bottlenecks holding it back, nor does it experience situations where a slowdown at one end causes overall queuing. Native liquid cooling systematically supports the computing power production line, keeping tokens highly efficient from generation to execution.
04.Conclusion: Hardware Convergence: System Architecture Capability Is the True Ceiling for Token Production Capacity
Currently, the hardware-level gap in the AI infrastructure track is narrowing. The core components that various manufacturers can procure are almost identical, and the performance ceiling of the chips themselves is becoming increasingly transparent. The GPUs you have, others have too; the HBM you use, others can also buy. Mere hardware stacking can no longer create a competitive edge.
As GPUs, HBM, and network chips gradually converge, and power and space become hard constraints for AI infrastructure, the true watershed lies in: who can organize these identical components into a more efficient computing power token production line.
The same scale of thousand-card computing power, placed in different rack system architectures, may result in a final token output difference of 30% or even more. The gap is not in the chips themselves, but in how chips communicate with each other, how heat is dissipated, how power is distributed, and how space is utilized... These system-level design choices ultimately determine how much of each chip's performance can be truly unleashed. This is where the value of system architecture capability lies.
It is the physical-layer definition of the upper limit of computing power conversion: What kind of cooling solution can keep chips running at full frequency continuously? What kind of interconnect topology can enable multi-card collaboration without creating communication bottlenecks? What kind of spatial layout can cram the most computing units into a limited rack? There are no standard answers to these questions; what is tested is the architectural design foundation of trade-offs and synergy among computing, cooling, interconnects, and power supply. Native liquid cooling is precisely the physical-layer embodiment of this system-level innovation.