Author | Yang Yiting, Editor | Bao Yonggang
In 2025, the golden age of the memory industry is unfolding.
TrendForce estimates that the global memory industry's output value will reach USD 889.3 billion this year, and will continue to grow by 53% in 2027, approaching USD 842.7 billion.
Amid the boom, the supply side is further locked down: the three major memory manufacturers—Samsung, SK Hynix, and Micron—have already finalized the capacity allocation for DRAM (Dynamic Random Access Memory) and HBM (High Bandwidth Memory) for the entire year of 2027 as early as August this year. Most customers can only secure 60% to 70% of the supply, and UBS predicts that the average price of HBM in 2027 will increase by over 79% year-on-year.
Such a pro-cyclical environment is hard to imagine for memory professionals in the past. But on the flip side of making money is a hurdle that the memory industry has never truly been able to overcome.
In the past, memory vendors' approach to dealing with demand growth was relatively linear: process evolution, capacity expansion, interface upgrades, capacity expansion, price hikes, and making money according to the cycle. The logic was not complicated. This time, when the demand surged in, the first thing it hit was the long-standing gap between memory and computing, known in the industry as the "memory wall."
"There is a river between memory and computing, and the memory wall will always exist. How to cross the river—whether to build a few small bridges or a highway—is the eternal proposition of memory architecture." described Chen Lun, an investor focusing on memory, regarding the relationship between memory and computing.
He is referring to a contradiction that has existed for decades but has been drastically amplified in the AI era: computing performance is growing rapidly, while the improvement in memory bandwidth simply cannot keep up.
As model parameters and context lengths continue to expand, this wall is getting wider. All current key memory directions are essentially answering the same question: how to cross the river?
01 The Golden Age, and the Old Rules Are Failing
In the past, memory vendors were not entirely passive in front of this wall. What supported them was a long-established industrial synergy mechanism: defining products through standards and meeting different market demands through scaled supply.
An important premise for this model to last for decades is that the demand side is sufficiently diversified. The development pace of markets such as PCs, mobile phones, and servers varies, making it difficult for a single customer to change the technological route of the entire industry based on its own needs. Therefore, the memory industry has formed a relatively stable cycle: standards are advanced according to industry consensus, vendors iterate products around the standards, and customers choose solutions based on their own needs.
But the emergence of AI is changing this balance.
The first thing to be broken is the pace of industrial evolution. Taking HBM as an example, from the release of the first-generation HBM standard (JESD235) in 2013, to the establishment of HBM2 in 2016, the launch of HBM3 around 2022, and the official implementation of the HBM4 standard in 2025, the evolution cycle of each generation of standards usually takes several years.
In the past, market demand was sufficiently dispersed, and the industry could always find a greatest common divisor through long-term negotiations. But in the AI era, the speed of change in computing architecture far exceeds the iteration cycle of traditional memory standards, and memory is beginning to face a new demand environment dominated by AI chip manufacturers.
Zheng Kai, a sales representative for AI memory solutions, believes that the problem lies in the fact that "NVIDIA launches a new generation of GPUs almost every year, while it takes JEDEC far more than a year to formulate a memory standard. Therefore, if HBM only produces standardized products, it will never keep up with the pace of GPUs."
When memory needs to be rapidly adjusted around GPU architecture, packaging methods, and system performance, the standardization path previously advanced through industry consensus begins to struggle to meet the iteration speed of AI infrastructure.
As a result, customized HBM is increasingly becoming an industry consensus: the product logic is shifting from "set standards first, then sell products" to "co-design around customer architecture". This change is already reflected in industrial practice: multiple memory vendors are participating in design and capacity planning in advance around the needs of leading AI chip companies, providing customized solutions for different architectures.
Zheng Kai compares this relationship to "tailoring a suit for a specific customer": the product can be closer to customer needs, but once customer needs change, the value of the dedicated design will also decline rapidly.
And the change does not only happen in the speed of product iteration.
In the past, when customers chose memory products, they focused on indicators such as capacity, performance, lifespan, and price. Standardized parameters could basically complete the procurement decision. But in the AI era, the questions raised by customers have gone beyond the scope of traditional memory specification sheets: In which layer of the memory architecture should the KV Cache in large model inference be placed? Can the GPU bypass the traditional path to directly access memory? Do memory devices need to undertake some computing tasks internally?
These questions are no longer focused on a memory product itself, but on how memory integrates into the entire AI infrastructure.
As Wu Zining, founder of Yingren Technology, judged: "The medium is no longer a standard product that passively adapts, but a component co-designed with the system architecture."
When memory changes from an independent product to a key component in an AI system, new questions arise: if memory needs to be co-designed with the system, who will take the lead in this redefinition?
02 Who Is Redefining Memory?
What AI Needs May Not Be Faster Memory Functions
The previous evaluation system for memory products was relatively clear: capacity, bandwidth, latency, lifespan, and price basically constituted the core indicators of a memory device. But AI is changing this evaluation logic.
In traditional scenarios, customers focus more on what performance a single device can provide; whereas in AI infrastructure, the value of memory increasingly depends on whether it can match the computing system. In other words, memory is no longer just about providing faster data reading and writing, but needs to ensure that data can be continuously supplied according to the computing pace of the GPU, avoiding expensive computing resources being idle while waiting for data.
The development of HBM is the most direct manifestation of this change, which is essentially the reverse traction of GPU computing power growth on the memory architecture.
Whether it is the large-scale parameter and gradient synchronization during the training phase, or the continuous generation of new tokens during the inference phase, the improvement in GPU computing power requires the memory system to provide higher data throughput. Especially in the large model inference process, as the context length and concurrent requests increase, intermediate data such as KV Cache continues to expand, and data movement is becoming an important factor affecting GPU utilization.
This is also why HBM continues to evolve towards higher bandwidth. According to JEDEC standards, the bandwidth of HBM3 reaches 819GB/s, while the HBM4 standard officially established in April 2025 further increases to 2TB/s, with the number of channels increasing from 16 to 32.
In just a few years, HBM bandwidth has increased by more than twice, but this growth has not ended the memory bottleneck. The reason is that the scale of AI models, context length, and inference concurrency are also growing synchronously, and computing demands are constantly re-consuming the newly added memory bandwidth.
If HBM represents AI's redefinition of "speed," then QLC represents AI's redefinition of "scale" and "efficiency." TLC (Triple-Level Cell) and QLC (Quad-Level Cell) are two types of NAND flash memory used in SSDs (Solid State Drives).
Ni Jinfeng, Vice President of Sales at Solidigm, mentioned that in the memory tiering of AI infrastructure, the fourth tier in the past usually undertook capacity-oriented and persistent data storage tasks, mainly handled by mechanical hard drives, and provided cache acceleration through front-end TLC SSDs. However, as the scale of GPU clusters expands and AI data volume continues to grow, the pressure of traditional solutions in terms of space occupation, power consumption, and O&M complexity is gradually emerging.
Therefore, some North American intelligent computing centers have begun to adopt large-capacity SSDs represented by QLC SSDs to replace some traditional capacity-tier solutions. Compared with mechanical hard drives, QLC SSDs can provide higher-density data storage in a smaller space, while reducing system power consumption and access latency.
For intelligent computing centers that need to carry tens of PB or even larger-scale data, under limited rack space and power budgets, it is more important how much effective capacity the memory system can provide, how much data access it can support, and what total cost of ownership it ultimately brings.
This change also explains why the value of memory in AI infrastructure is being re-evaluated.
From the perspective of the construction phase, GPUs usually account for the majority of the investment in intelligent computing centers, and memory investment is relatively limited. But entering the operation phase, whether the memory system is efficient will directly affect GPU resource utilization. If data cannot be supplied to computing nodes in a timely manner, even the most powerful GPU may be in a waiting state.
Therefore, the standard for measuring the value of memory in the AI era is changing. Wu Zining, founder of Yingren Technology, proposed that more important indicators in the future may be "power consumption per TB" and "performance per watt": the focus of memory competition is not just how fast a single device can run, but how to support more effective computing with lower energy consumption in the entire AI system.
Once, the performance of a disk was mainly determined by its own parameters; now, what kind of memory is considered high-performance needs to be re-measured in the context of GPU architecture, data flow methods, and AI workload demands.
How Data Flows Is More Important Than Where Data Is Placed
After the change in the performance evaluation system, the problem further extends to the system level: when a memory device no longer only needs to provide fixed capacity, bandwidth, and latency, how data actually flows between the GPU, memory, and storage also begins to become part of the memory value.
In the past, there was a relatively mature division of labor between memory and upper-layer systems. Memory devices provide capacity and performance through standard interfaces such as SATA and NVMe, while the host, operating system, and application software are responsible for how to use these resources.
For databases, file systems, and ordinary applications, this division of labor is already sufficiently effective.
What AI changes is data access itself. Large model inference is not simply reading data from memory. As the context continues to grow, the capacity of the KV Cache continues to expand; different models and different inference frameworks also have different ways of generating, migrating, and reusing the KV Cache.
Where data is placed, when to migrate it, which data needs to stay in high-bandwidth memory, and which can be sunk to lower-cost memory tiers are gradually becoming important issues affecting inference efficiency.
This means that memory is beginning to face a batch of new demands that cannot be directly described by traditional parameter sheets.
Mooncake by Yuezhi Anmian is a typical case of this change. In the underlying inference system of Kimi, Mooncake splits Prefill and Decode into two relatively independent resource pools: the former mainly processes input prompts, which is compute-intensive and easier to parallelize; the latter is responsible for token-by-token generation, which is more sensitive to memory access and data supply.
On this basis, Mooncake further organizes CPUs, DRAM, SSDs, and high-speed networks to build a unified KV Cache resource pool and manages it in tiers according to data access conditions.
SSDs are placed in a completely different data flow. Although the capabilities of the medium are fixed, the way it is accessed, the logic of its scheduling, and the standards by which it is evaluated are all redefined by the upper-layer inference system.
The problem then falls to a new industrial link: how to truly implement the data access demands proposed by upper-layer software down to the underlying medium?
Wu Zining calls this process a layer of "translation."
Memory manufacturers are mainly responsible for the technological evolution of the medium itself, such as NAND; model companies and cloud vendors define new data access methods from the application and system levels. In between, someone needs to understand the characteristics of different mediums and translate the access semantics proposed by upper-layer software into specific device behaviors.
Controller manufacturers are taking on more and more of such work. The most core task of controller chips in the past was to manage flash memory particles from different manufacturers and different generations, encapsulating the complex underlying physical characteristics into stable and predictable SSD products.
And as AI systems put forward more differentiated demands on memory, controller firmware is also beginning to become an important connecting layer between the upper-layer system and the underlying medium.
The "translation" mentioned by Wu Zining essentially describes this change: the upper-layer system decides what kind of data access methods are needed, and the controller needs to understand these demands, combine them with the characteristics of the medium, and implement them into specific data scheduling, access, and management mechanisms.
And when this collaboration continues to expand outward from a single memory device, the problem enters a larger system scope.
The "integration of computing, memory, and transmission" proposed by Zhongke Shuguang points to exactly the same change: the originally relatively independent resource boundaries between computing, memory, and network are being reorganized in the same system.
The Zhongke Shuguang 8000 100,000-card AI super cluster achieved full-load operation in the first week of going online, with the average daily processed jobs exceeding 150,000, demonstrating the practical effect of this system-level collaboration.
The demand for this collaboration does not stop at the coordination between memory and computing. When the cluster scale continues to expand, the physical boundaries between resources begin to become efficiency bottlenecks, and the reconstruction of the interconnection layer becomes inevitable. The server-centric tightly coupled architecture is being decoupled into a larger-scale resource pool.
CXL provides a generalized implementation path. Based on the PCIe ecosystem, it uses memory semantics to achieve unified addressing and efficient access between CPUs, memory, and other devices, allowing devices from different manufacturers to access the same memory pool. Interestingly, memory manufacturers are not very positive about CXL. The three major manufacturers—Samsung, SK Hynix, and Micron—have simultaneously halted their self-developed commercialization plans for CXL controllers.
The reason is that self-developed integrated solutions may cannibalize their core DRAM module business. If CXL turns memory into a poolable resource, customers will not need to buy so many high-priced dedicated memory modules. This market gap is being filled by third-party chip companies such as Lanqi Keji and Astera Labs.
When computing, memory, storage, and network begin to be designed as a whole, the role of memory also changes accordingly.
It still needs to make continuous progress in medium density, bandwidth, latency, and reliability, but how these capabilities are used increasingly depends on upper-layer data flow design, controller firmware implementation, and system-level resource orchestration.
The originally relatively direct chain of the memory industry—medium, device, host—is being lengthened. Models and applications propose new access demands, system software decides how data flows, and the controller and device layers are responsible for translating these demands into executable mechanisms, which finally fall to the underlying medium.
In this process, new value is generated not only in the memory medium itself but also at the connections between different layers. Whoever can more effectively transmit the demands of upper-layer AI workloads to the underlying hardware will have the opportunity to become a new key node in the AI memory industry chain.
The Next Piece of Cake in Memory, Not Necessarily Belonging to Memory Vendors?
As data flow and resource architecture are reorganized, AI pushes the problem one step further: when data movement itself increasingly becomes a burden on performance and power consumption, how much computing should actually enter memory?
The answer to this question determines not only the chip architecture but also how far memory vendors can move forward in the AI computing chain.
The closer the computing unit is placed to the data, the more likely it is to theoretically reduce the bandwidth and power consumption losses caused by repeated data movement between different tiers. But once computing power enters memory, it will also change the original division of labor in chips: memory vendors need to decide which computing is worth undertaking by themselves, and which parts should still be left to dedicated computing chips.
Around this boundary, two different promotion methods are forming in the industry.
The first path is Processing-In-Memory (PIM) promoted by memory manufacturers.
Companies such as Samsung and SK Hynix have been continuously promoting PIM in recent years, hoping to directly add a certain amount of computing power inside memory such as HBM, allowing some data-intensive tasks to be completed near the data and reducing repeated data transmission between the GPU and memory.
Samsung once publicly disclosed that solutions equipped with HBM-PIM can reduce energy consumption by about 2,100 gigawatt-hours per year compared to systems using only HBM.
But memory manufacturers are relatively restrained in promoting "computing entering memory."
Chen Gang, head of the advanced memory-computing technology R&D department at Houmo Zhineng, believes that this is related to the business model of the DRAM industry. DRAM manufacturing relies on dedicated processes, and manufacturers usually adopt a highly integrated IDM model. Huge manufacturing and R&D investments need to be amortized through large-scale shipments. Therefore, when computing power is added to DRAM, manufacturers need to consider not only whether the technology is feasible but also whether this capability can cover sufficiently broad demands and maintain the original scale effect of the product.
This determines that PIM is more likely to start with computing tasks that have strong commonality.
Matrix-vector multiplication such as GEMV is a typical example. It is highly related to the data access characteristics in the inference decoding stage of large models, and the computing mode is also relatively clear. For memory manufacturers, pushing such tasks to the memory side can reduce some data movement without requiring DRAM to undertake complete general-purpose computing functions.
If computing continues to be pushed towards data, new players outside of memory manufacturers are also beginning to enter this space.
New chip companies such as Zhicun Keji, Houmo Zhineng, and Guangyu Xinxingchen choose to cut in from the chip architecture itself. They explore different technological routes such as SRAM, DRAM, and RRAM. The goal is not simply to add a processing unit to traditional memory, but to rearrange the relationship between computing units and memory arrays, allowing more operations that originally needed to be completed in computing units to be completed directly near the data.
For the memory industry, the significance of this change lies in: when "reducing data movement" becomes a new computing demand, memory manufacturers do not naturally possess this newly added value.
The advantage of manufacturers still lies in the medium and manufacturing capabilities, but new entrants can rely on architectural innovation to undertake some functions that originally belonged to GPUs, NPUs, or memory devices in specific scenarios.
What truly determines the competitive space for the two types of players is exactly where in the data link the computing is pushed. The closer to the data, the more it can theoretically reduce movement, but the easier it is to bind with specific algorithms and applications. For memory manufacturers, this means gradually increasing computing capabilities while maintaining the universality of memory products; for new chip companies, it means making trade-offs between stronger scenario targeting and software ecosystem and commercialization costs.
Ultimately, the leading power of the industry chain may also therefore gradually shift from "who manufactures memory" to "who can design new computing-memory architectures."
Wu Zining judges that manufacturers can continue to deeply cultivate medium innovation, which is the most irreplaceable capability of memory, while controller manufacturers hold the "translation layer" connecting different mediums, protocols, and systems. In this division of labor, memory does not need to undertake more and more computing functions, but rather makes medium management and system adaptation deep enough to hold its own value boundary.
And it is precisely in this sense that the "golden age" is no longer just a judgment about market size and the boom cycle.
It is not only the golden age of memory demand but also the era when the memory industry is re-finding its own position. The real change is not whether memory will continue to grow, but who can translate this importance into their own industrial discourse power after memory becomes increasingly important.
This is perhaps the true proposition left by AI to the memory industry: market expansion is only the first step; how to expand one's own value boundary while the market expands.