Author: Junxi Recently, NVIDIA announced on its official blog that its next-generation AI computing platform, Rubin, will completely eliminate fans, relying 100% on liquid cooling. In the same month, a team from the Korea Advanced Institute of Science and Technology (KAIST) published a paper demonstrating a cooling technology that injects room-temperature water directly into microchannels inside the chip, achieving a coefficient of performance (COP) 10 times the previous world record. A few days prior to that, SK Hynix unveiled the iHBM solution, which directly integrates heat dissipation components into HBM memory packaging.
The dense emergence of these messages is no coincidence. As the power consumption of a single AI accelerator approaches 1000W and the power of a single rack nears 1 megawatt, the physical limits of air cooling have been reached. The judgment from the president of Schneider Electric is straightforward: "Once the power consumption of a single chip exceeds a certain threshold, liquid cooling is no longer an option, but a necessity."
Centered around "heat dissipation," the semiconductor industry is undergoing a full-chain reconstruction from the inside of the chip to the exterior walls of data centers.
01. KAIST's Disruptive Breakthrough In the field of chip-level heat dissipation, traditional external liquid cooling solutions are facing bottlenecks such as high fluid resistance, high pumping energy consumption, and uneven temperature distribution. On June 16, the KAIST research team published a breakthrough study demonstrating an ultra-efficient liquid cooling technology that cools the chip from the inside.
Instead of relying on expensive special thermal conductivity materials like synthetic diamond, the KAIST team etched the "manifold microchannel" structure directly inside the silicon semiconductor chip. This design is similar to a highly efficient logistics network. By evenly distributing multiple micro-inlets and outlets on the chip, it significantly shortens the transmission distance of the cooling fluid, thereby substantially reducing fluid resistance and the required pumping pressure.
The core advantages of this technology are reflected in three dimensions: First, extremely high cooling efficiency—in experiments, the system achieved a cooling coefficient of performance (COP) of 106,000 — 10 times the previous world record reported in Nature in 2020, meaning chip manufacturers can remove the same amount of heat with only one-tenth of the pumping power. Second, stability under extreme thermal loads—even under an extreme thermal load of 2000 watts per square centimeter, the system can still use ordinary room-temperature water to keep the chip temperature below the safe threshold of 100°C. Third, compatibility with existing processes—the entire microchannel manufacturing process is completed below 350°C, fully compatible with existing commercial semiconductor wafer foundry production lines, without the need to spend billions of dollars on new equipment.
KAIST Professor Sung Jin Kim pointed out that as the performance of AI semiconductors and advanced electronic packaging becomes increasingly limited by heat, this technology is expected to become the foundational cooling solution for future high-performance computing systems. The commercialization prospects for microfluidic chip cooling technology are broad. According to a report by Fact.MR, the global microfluidic chip cooling market will surge from $384.3 million in 2025 to $2.86 billion in 2036, with a compound annual growth rate (CAGR) of up to 20%.
02. The "Thermal Defense Battle" in the HBM5 Era In AI computing systems, data transmission between the computing core (GPU/ASIC) and high bandwidth memory (HBM) is critical for performance. However, as HBM advances from HBM3E to HBM4E and even HBM5, with the number of stacking layers expected to reach around 20, heat accumulation has become a core bottleneck limiting performance and scalability. The focus of competition among the top three memory chip giants—SK Hynix, Samsung, and Micron—has shifted from a simple comparison of capacity and bandwidth to a contest in packaging-level thermal management technology.
On May 27, SK Hynix took the lead in releasing the "iHBM" thermal solution, announcing its application in next-generation products including HBM5. Traditional HBM designs rely on heat dissipation through the base die, whereas SK Hynix's iHBM solution fundamentally revolutionizes the structure. This technology directly integrates cooling components into the D2D PHY interface between the HBM stack and the GPU. ICE is a silicon-based material with high thermal conductivity and electrical insulation, creating an extra heat dissipation channel within the package. Official data from SK Hynix shows that this design can reduce thermal resistance by 30% while significantly improving the system's operational stability under high loads.
Not to be outdone, Samsung Electronics showcased its HBM5 model equipped with HPB technology at the subsequent COMPUTEX 2026 exhibition. Song Jae-hyuk, Chief Technology Officer of Samsung's DS Division, confirmed that HPB technology has been implemented in HBM4E, and its reliability has been verified. Similar to SK Hynix, Samsung also targets the D2D PHY area, a major heat source. HPB technology introduces an independent silicon-based thermal path in the D2D PHY area to improve heat conduction. Samsung previously applied a copper-based HPB structure to its Exynos 2600 application processor, achieving a thermal resistance reduction of up to 16%. In HBM applications, Samsung is committed to integrating HPB into the global design of the entire memory stack, optimizing the layout of the base die and core dies.
Micron Technology, on the other hand, has adopted a different technical route. Micron focuses on low-power HBM design, primarily achieved through its Through-Silicon Via (TSV) trench cooling technology. This technology etches microscopic trenches inside the silicon wafer of AI accelerator chips and circulates coolant to reduce internal heat accumulation. In addition, a U.S. patent obtained by Micron in 2025 reveals a vertical thermal management structure based on electrically passive cooling TSVs. These dedicated heat dissipation TSVs are located within the same package pins as signal TSVs, occupying no additional die area and forming a low-resistance vertical heat dissipation channel.
The fierce competition among HBM manufacturers in thermal management indicates that advanced packaging technology has transcended the realm of pure electrical interconnects, taking the heat conduction path as a core element of architectural design. This shift will profoundly impact the packaging yield and manufacturing costs of future AI chips.
03. Reconstruction of NVIDIA's Rubin Platform If KAIST and memory manufacturers are solving heat dissipation problems at the chip and packaging levels, NVIDIA is driving an infrastructure revolution at the system and data center levels. On June 21, 2026, NVIDIA's official blog published an article detailing the thermal architecture of its next-generation Rubin platform.
The Rubin platform is the world's first 100% fully liquid-cooled AI computing platform. In Rubin servers, not only the GPUs and CPUs, but all networking components are completely cooled by a closed-loop liquid cooling system, completely eliminating fans within the system. The core breakthrough of this design lies in the operating temperature of its coolant (a mixture of 75% water and 25% propylene glycol). The inlet temperature of traditional coolants is usually around 30°C, whereas the Rubin system pushes the coolant inlet temperature up to 45°C, reaching approximately 55°C when it flows out of the server.
Raising the coolant temperature is based on a fundamental principle of physics: heat flows from high-temperature objects to low-temperature objects. The higher the temperature of the coolant when it reaches the outdoor dry cooler, the easier it is for the passive outdoor dry cooler to dissipate heat without relying on mechanical chillers or evaporative cooling towers. According to industry estimates, for every 1-degree increase in chiller temperature, cooling energy consumption costs can be reduced by approximately 4%.
Ali Heydari, Director of Data Center Cooling and Infrastructure at NVIDIA, stated: "The DSX reference design achieves zero water consumption. Except for the 1% of the time when chillers might be needed under certain climate conditions, this is almost a closed-loop system without the need for evaporative cooling." For a 50-megawatt hyperscale data center, switching to this liquid cooling infrastructure can cut annual cooling energy and water costs by more than $4 million combined. Furthermore, the fully liquid-cooled architecture significantly increases rack density; a system that originally occupied 6 rack units now requires only 2 units, while eliminating the noise of up to 85 decibels or more from traditional air-cooled servers.
This move by NVIDIA has strong industry appeal. Since the Rubin platform adopts a fully liquid-cooled design, all cloud service providers (CSPs) and data center operators building systems for this platform must complete the transition to liquid cooling technology. Server manufacturers such as Dell and Supermicro have responded swiftly. Dell launched the fanless, direct liquid-cooled PowerEdge XE8812 server, which can accommodate 144 GPUs in a single rack with a power output exceeding 300kW. Supermicro, on the other hand, partnered with ExxonMobil to validate immersion cooling technology based on NVIDIA B300 AI servers and delivered an end-to-end Rubin NVL4 liquid-cooled rack solution.
04. The Golden Age of Liquid Cooling Startups As liquid cooling technology becomes an absolute necessity, capital market attention to this sector has reached unprecedented heights. Recently, liquid cooling startups have frequently announced massive funding rounds.
Accelsius announced the completion of a $65 million Series B funding round, led by building technology giant Johnson Controls. The company's NeuCool two-phase direct-to-chip liquid cooling platform adopts a waterless design, which is claimed to save up to 50% in energy compared to traditional systems, with a cooling capacity of over 4500W per slot.
Another highly watched startup is Omen AI. The company completed a $31 million Series A funding round led by Nava Ventures at the end of June. As liquid cooling systems become widespread, the health condition of the coolant has become a hidden danger affecting the stable operation of data centers. Omen AI has developed a micro-spectrometer that uses artificial intelligence to monitor the chemical composition of the coolant in real time, issuing warnings before bacterial outbreaks or equipment wear lead to millions of dollars in downtime losses. Currently, the company has partnered with more than a dozen data center clients, including TensorWave.
In the secondary capital market, investors have also voted with their hard-earned money in favor of the liquid cooling trend. Following NVIDIA's release of Rubin liquid cooling details, traditional HVAC (Heating, ventilation, and air conditioning) stocks fell sharply. This reflects the market's belief that the market share of traditional air cooling equipment in the AI data center market will be rapidly eroded by liquid cooling technology. Meanwhile, companies with deep layouts in the liquid cooling field, such as Vertiv and Schneider Electric, have seen significant growth in their market capitalization over the past year. In a June research report, BNP Paribas listed Vertiv and Eaton as top picks in the AI data center cooling sector.
05. Challenges Beyond the Boundaries Although liquid cooling technology has shown great potential in reducing energy and water consumption inside data centers, it is not a "panacea" for solving the AI energy crisis.
Andrew Chien, a computer science professor at the University of Chicago, pointed out that NVIDIA's 45°C closed-loop system is indeed an engineering marvel, but the so-called "zero water consumption" is merely a statistical result within the physical boundaries of the data center. According to analyses by Xylem and Global Water Intelligence, by 2050, water directly consumed for data center cooling will make up roughly 4% of the total new water demand created by AI. By comparison, power plants that supply power to data centers will account for 54% of this new water demand, and semiconductor manufacturing will account for 42%. This means that although liquid cooling technology solves the "immediate cooling problem at the data center site", it does not fundamentally eliminate the massive consumption of overall water resources and energy by the AI industry chain.
Furthermore, the geographical environment is also a constraining factor for the popularization of liquid cooling technology. NVIDIA's 45°C system can operate without chillers in temperate climates, but in hot regions such as Arizona, Texas, or Singapore, it still relies on mechanical cooling on the hottest days. Yet, a large number of currently planned AI data centers are located precisely in these water-stressed regions.
In terms of commercial deployment, advanced technologies such as immersion cooling still face challenges regarding maintenance complexity. Submerging servers in dielectric fluid means that hardware maintenance requires lifting the equipment out of the cooling tank, draining the fluid, and cleaning it, which greatly increases the time and difficulty of operations and maintenance. Japanese telecom giant KDDI, in partnership with Mitsubishi Heavy Industries, deployed a commercial data center using immersion cooling in Sakai, Osaka, reducing cooling energy consumption by 94% and lowering the PUE to 1.05. However, the upfront capital expenditure for such deployments is much higher than that of air cooling systems, and retrofitting existing legacy data centers is extremely difficult.
The "Jevons Paradox" in economics also applies here: when the cost of cooling every watt of computing power becomes lower and easier, the most likely outcome is the deployment of more and denser computing power, thereby offsetting the savings in unit energy consumption at the system level.
06. Conclusion Chip cooling has leaped from a marginal engineering support link to a core strategic element determining the success or failure of AI infrastructure. From KAIST's microchannel innovation, to the packaging-level thermal defense by SK Hynix and Samsung, to the rack-level fully liquid-cooled revolution led by NVIDIA, a clear technological evolution path has emerged: cooling systems are continuously moving closer to the heat source (the silicon die).
In this new era driven by computing power, whoever can manage heat most effectively will gain an advantage in performance, density, and operating costs. Thermal management is no longer just a physics problem; it has become the "New Moore's Law" of the AI era, defining the physical boundaries and commercial ceiling of computing power growth. For participants in the semiconductor industry chain, mastering advanced cooling technology means holding a crucial ticket to enter the future AI computing power landscape.