Author | Chen Yuelin Editor | Bao Yonggang
The AI server memory bill, long dominated by HBM, is being rewritten by "mobile memory."
On August 11, foreign media outlet Wccftech, citing estimates from Bank of America (BofA), reported that NVIDIA's Kyber rack for the Vera Rubin Ultra NVL144 will be equipped with 124.4TB of HBM4E, costing approximately $2.5 million. Meanwhile, the 216TB of LPDDR5X in the same rack, despite a lower unit price, will bring the total cost close to $2.8 million, exceeding the former by about 12%.
LPDDR5X was originally primarily used in mobile devices such as smartphones; for instance, the iPhone 17 Pro is equipped with 12GB of LPDDR5X memory. To put this into perspective, the amount used in a single Kyber rack is equivalent to the total running memory of approximately 18,000 smartphones.
This is not the first time LPDDR has entered the data center. As early as the Grace CPU era, NVIDIA had already begun using LPDDR5X to replace the DDR traditionally used in servers.
Three years later, the memory for the new-generation Vera CPU has not only changed in form—SOCAMM2 (Small Outline Compression Attached Memory Module 2) has upgraded the original fixed design to a detachable and maintainable server memory module—but the LPDDR5X capacity per CPU has also increased from the GB level to the TB level (1TB = 1000GB).
This product, originally designed for edge devices like smartphones to balance bandwidth, capacity, and power consumption, is now facing a situation similar to HBM—becoming increasingly expensive while also becoming increasingly scarce.
01. Why Can Former Mobile Memory Run in AI Servers?
The answer lies in the continuous increase in CPU core counts and the ever-growing data volume for workloads such as intelligent computing and supercomputing. These pressures are transmitted to the memory side, translating into a clear demand for larger capacity and higher bandwidth.
However, DDR5 RDIMM (Registered Dual In-line Memory Module), traditionally used for server main memory, struggles to meet the CPU's pursuit of ultimate memory throughput while controlling power consumption.
Consequently, NVIDIA began to explore alternative approaches, targeting low-power mobile memory like LPDDR. The LPDDR5X, first applied in the inaugural data center CPU Grace, features multiple memory channels arranged around the CPU, providing total bandwidth far exceeding that of smartphone scenarios.
According to official data disclosed in 2023, compared to typical server DDR5 memory solutions at the time, Grace's LPDDR5X can increase bandwidth by up to 53% at a similar cost; to provide the same 1GB/s bandwidth, the required power consumption is only about one-eighth of the former.
The latest joint tests by Micron and Meta more intuitively demonstrate how bottlenecks are resolved by LPDDR5X:
The throughput of an AI data curation task increased by 11% after the memory speed was improved; when the capacity was doubled, avoiding frequent writes of intermediate data to SSDs, the speed of big data analysis tasks even increased to 2 to 3 times the original; in one of the AI tasks, LPDDR5X itself accounted for only 6.8% of the total system power consumption.
When mobile memory is brought into servers, the balance between performance and power consumption can be maintained. But the next question arises: the installation methods of traditional mobile devices may not necessarily adapt to the working rhythm of servers.
To shorten signal paths and reduce transmission losses, Grace fixed LPDDR5X onto the CPU compute module.
This design often means that once LPDDR5X is damaged, the entire compute board carrying the memory needs to be replaced, which does not align with the characteristics of servers that require continuous operation for years and the ability to quickly replace faulty components on-site.
In addition, flexibility on the supply side is also weakened. Given that OEMs must lock in memory capacity and die specifications in advance, delivered servers are difficult to upgrade iteratively later. Once the supply of a certain die becomes tight, manufacturers also find it hard to adjust configurations as freely as they would when replacing RDIMMs.
Thus, SOCAMM, which modularizes LPDDR5X, emerged to meet this need.
However, it did not inherit the layout concept of DDR5 RDIMM, but instead drew on the CAMM (Compression Attached Memory Module) design used in thin and light laptops:
Traditional RDIMMs use vertical slots. Although they are convenient to remove and replace, the tall memory sticks occupy three-dimensional space around the CPU;
The flat compression-attached solution not only retains removability but also significantly reduces the module height, leaving space for cooling components around the CPU and more LPDDR5X modules.
According to Micron's estimates, at the same capacity, the first-generation SOCAMM can provide more than 2.5 times the bandwidth of RDIMM, occupying about one-third of the area of the latter; the power consumption of its successor, SOCAMM2, is also about one-third of that of an RDIMM with the same capacity.
Under the constraint of total rack power consumption, the low-power advantage of SOCAMM is being further amplified.
Official NVIDIA data reveals that the four GPU modules (including compute and their accompanying HBM) in a GB200 compute node have a combined maximum power consumption of 5.6kW, accounting for approximately 84% of the node's 6.7kW maximum power budget.
When GPU modules have already taken up most of the power budget for AI racks, the expansion of CPU memory has to be meticulously calculated, and SOCAMM hits the pain point exactly. Therefore, it is not hard to understand why NVIDIA has continuously bet on the CPU memory track from LPDDR to SOCAMM.
However, the performance advantages of high bandwidth, low power consumption, and large capacity only demonstrate the feasibility of this route, but cannot explain the other side of what is happening—
As AI systems consume TB-level amounts of LPDDR and more manufacturers successively enter the fray to bet on it, the logic where prices and inventory were primarily driven by the smartphone cycle in the past is beginning to change.
02. When Only 60% of Demand Is Met, NVIDIA May Cut Configurations
The expansion of LPDDR on the CPU side has been directly written into NVIDIA's new-generation product specifications.
The Grace CPU, which began commercial use three years ago, features a maximum configuration of 480GB of LPDDR5X per chip; by the time of Vera, this figure has increased to 1.5TB, more than three times the former.
In the entire system, the 36 Vera CPUs in the Vera Rubin NVL72 are configured with a total of 54TB of LPDDR5X, while BofA's estimate for the next-generation Kyber rack further reaches 216TB.
The continuous increase in capacity is not just for padding specifications, but to match the increasingly heavy workloads of computing systems.
With the rise of agents, the tool calls, code execution, and database access that CPUs need to handle are no longer one-time actions but occur repeatedly within a single task, accompanied by an increase in parallel tasks.
CPUs need to process and move more data, naturally raising the requirements for memory capacity and bandwidth.
The division of memory labor between CPUs and GPUs is also simultaneously applying pressure.
Through coherent interconnects, some of the capacity pressure originally concentrated on the GPU side is transferred to the CPU side, while the limited and expensive HBM space is reserved for data that is more sensitive to bandwidth.
When both demands ultimately point to larger memory capacity and higher transmission bandwidth, memory manufacturers are also following this trend to continue increasing SOCAMM capacity.
Currently, the top three DRAM manufacturers have all entered the mass production stage of SOCAMM2. Micron has also adopted a single 32Gb (approximately 4GB) LPDDR5X die to further increase the capacity of a single SOCAMM2 module to 256GB, and is currently in the customer sampling stage.
NVIDIA is also not the only chip manufacturer starting to expand the use of LPDDR on server CPUs.
In April this year, AMD announced plans to launch CPUs supporting LPDDR5X SOCAMM2 in 2027, positioning them as host companions for future Instinct GPUs.
TrendForce also noted that ASIC developers other than NVIDIA have begun evaluating low-power DRAM.
It is estimated that between 2028 and 2030, the AI server ecosystem may even surpass smartphones to become the world's largest single end-market for LPDRAM (a general term for all low-power memory products, including LPSDRAM, LPDDR4, LPDDR5, LPDDR5X, etc.).
SOCAMM itself is also transitioning from a solution pioneered by NVIDIA to a broader industry standard.
In June this year, JEDEC, the global standard-setting body for memory and solid-state technology, officially defined the electrical and mechanical requirements for SOCAMM2. This means more manufacturers can develop modules and platforms according to unified specifications.
However, while new buyers have not yet entered the market on a large scale, the supply of LPDDR5X, which SOCAMM relies on, is already tightening.
In January this year, TrendForce pointed out that DRAM manufacturers are shifting more of their advanced nodes and new capacity toward server memory (mainly referring to the traditional DDR5 route) and HBM, thereby narrowing the supply space for other products like LPDDR.
In addition, even though the smartphone market is in its traditional off-season, LPDDR5X remains in short supply, and prices continue to rise.
This means that SOCAMM, which shares the same origin, is also unlikely to escape this fate.
SemiAnalysis also pointed out that LPDDR5X converted into server modules must undergo more complex product development and longer qualification cycles.
Based on this, according to further estimates from its Memory Model, NVIDIA's contract price for procuring SOCAMM in the first quarter of 2026 is approximately $8/GB, a significant increase from the fourth quarter of 2025, and also higher than the approximately $6 to $7/GB for mobile LPDDR5X during the same period.
SemiAnalysis also predicts that by the end of 2026, the price of SOCAMM may exceed $13/GB.
Price increases are only the first layer of impact. When the limited LPDDR5X is insufficient to fill all Vera CPUs, the supply challenge will further evolve into an allocation problem.
Faced with tight supply, even NVIDIA has to consider halving SOCAMM. As early as June 4, SemiAnalysis reported that most Vera Rubin systems might adopt 96GB instead of 192GB of SOCAMM2, reducing the memory per Vera CPU from a maximum of 1.5TB to 768GB.
A day later, when asked about this, Jensen Huang only acknowledged limited memory supply and stated that NVIDIA needs to allocate memory more reasonably across different systems.
On June 10, TrendForce further quantified this gap. According to the preliminary 2027 supply allocation from Samsung, SK Hynix, and Micron, the LPDRAM NVIDIA can obtain will only meet about 60% of the expected demand.
According to its research, to allow the limited LPDDR5X to support the shipment of more Vera CPU systems, NVIDIA chose to halve the SOCAMM capacity configured in Vera Rubin racks.
Two months later, TrendForce reiterated this judgment in another report, stating that the tight supply of LPDDR5X may last until 2027.
Currently, NVIDIA has not publicly responded, but multiple rounds of news regarding Vera configurations have sent a signal—the memory discussion, previously primarily triggered by HBM on the GPU side, is extending to the CPU side and beginning to affect the final configuration in which the entire AI system is delivered.
Memory is still constraining the expansion of AI computing systems.