Author | Yang Jingli, Editor | Li Shuiqing
According to a report by Zhidx on September 17, on September 1, MLCommons, the global authoritative AI performance evaluation organization, released the MLPerf Storage v3.0 benchmark leaderboard. The YanRong All-Flash AI Storage F9000X secured first place in multiple benchmark tests, fully validating its highly efficient utilization capability of high-speed network bandwidth under extreme AI training workloads.
As the AI industry shifts from model training to large-scale inference, the data processed and tasks undertaken by storage systems are also increasing. During the training phase, storage must continuously supply massive amounts of data to GPUs and quickly save and restore model checkpoints; during the inference phase, the system must manage the ever-expanding KV Cache. A single high-performance storage product can no longer cover the full-process requirements of AI. Focusing on AI training and inference scenarios, YanRong has built a relatively complete full-stack AI storage solution. The test results of the F9000X fully validate its training-side capabilities: the newly added KV Cache test case in MLPerf Storage v3.0 further expands its coverage of AI full-scenario workloads.
The YRCache ContextBox appliance recently launched by YanRongTech is a dedicated KV Cache storage product for AI inference scenarios, representing a further implementation of its full-stack AI storage capabilities in the direction of KV Cache management, sharing, and reuse. From high-speed read/write of training data to the sharing and reuse of inference contexts, how does YanRong address the storage needs at different stages of AI? What performance and cost values can this full-stack AI storage solution bring to enterprises? This article provides an in-depth analysis.
01.Three-Node Bandwidth Sets New Record Ensuring High-Efficiency Read/Write Performance
Large model training requires continuous data reading from the storage system. As the number of GPUs participating in training increases, the storage system needs to supply data to more computing nodes simultaneously. If the data throughput capacity is insufficient, GPUs may experience reduced utilization due to waiting for data. Checkpoints also need to be written periodically during training, and their read/write performance directly affects the time taken to save model states and recover after training interruptions. The YanRong F9000X is an all-flash storage product launched for large-scale AI training, and its training-side strength can be verified through the MLPerf Storage benchmark tests. The test environment includes three F9000X storage nodes and nine x86 client servers, with nodes interconnected via an NDR400 high-speed network.
The 3D U-Net model training test poses high requirements for the continuous bandwidth output capability and extreme load stability of the storage system. In the test, the F9000X three-node cluster achieved an aggregated bandwidth of 544GiB/s, ranking first in this scenario and reaching the highest level among all publicly released test results. Compared to the 478GiB/s in the previous v2.0 test, its aggregated bandwidth has further improved, indicating that the storage system can provide training data to GPUs more efficiently.
In the Checkpoint70B test, the YanRong storage cluster achieved a read bandwidth of 539GiB/s and a write bandwidth of 314GiB/s, with both read and write performances ranking first, setting new records in all historical public tests of MLPerf Storage. For hundred-billion and trillion-parameter model training, checkpoints are critical data for saving phased training states. Faster read/write speeds can reduce the time consumed by model archiving and recovery, lowering the computing power waste caused by training interruptions.
For enterprises, the higher bandwidth performance of the YanRong F9000X can not only accelerate the read/write of training data but is also expected to reduce storage node investments while meeting the same training requirements, thereby lowering overall hardware investment costs.
02.AI Enters the Inference Phase Storage Begins to Manage Model Contexts
As AI enters the inference phase, the focus of storage systems shifts to the management and reuse of model contexts. Long texts, multi-turn dialogues, and Agent applications generate massive amounts of KV Cache. If these caches cannot be saved and reused, the model will need to repeatedly compute the same contexts. Meanwhile, inference requests feature longer contexts and higher concurrency, requiring KV Caches to be shared, migrated, and reused across different nodes. To address this issue, YanRong launched the AI-native inference storage system YRCache, which uniformly manages KV Cache resources at different tiers and offloads part of the cache from GPU memory to host memory and local SSDs to reduce redundant computations.
In the multi-tier cache architecture of YRCache, G1 is the GPU HBM, responsible for saving the most active and frequently accessed KV Cache currently; G2 is the host DRAM, used to accommodate caches that temporarily cannot be placed in GPU memory; G3 is the local SSD, which offers larger capacity and lower unit cost, suitable for saving more caches within a single node; and G4 is the traditional shared distributed storage, primarily bearing data that needs long-term preservation, such as models, datasets, and checkpoints.
As inference clusters expand, a new capability gap emerges between local SSDs and traditional shared storage. Referencing the design of G3.5 Context Memory in the NVIDIA CMX architecture, YanRong introduced a G3.5 shared cache tier into the YRCache system and launched the YRCache ContextBox appliance. It provides cross-node shared KV Cache space for long-context, multi-turn dialogue, and Agent inference, supporting cache sharing, reuse, and independent scaling while being close to the computing resources.
03.Filling the Shared Cache Tier Enabling Cross-Node Reuse of KV Cache
Designed for the G3.5 shared cache tier, the YRCache ContextBox integrates computing, networking, storage, and software to provide an independent shared KV Cache space for inference clusters. According to YanRong Technology, a single ContextBox unit achieves a measured bandwidth of up to 120GB/s, supports concurrent access from eight inference nodes, and offers cache capacity scalable to the PB level.
First, the YRCache ContextBox enables different inference nodes to share already generated KV Caches. Even if a request is scheduled to another server, there is no need to recompute the same context. Meanwhile, the cache is no longer limited by the GPU memory, host memory, and SSD capacity of a single server, allowing enterprises to scale the cache space independently; even if computing nodes are added or removed, the saved caches will not be lost.
According to YanRong Technology, in a test at a leading AI enterprise, eight NVIDIA H20-3e GPUs running SGLang and GLM-5.1 with input contexts ranging from 31K to 129K achieved an 89% to 94% reduction in first-token latency and a 3x to 6x increase in token throughput after connecting to the ContextBox. The ContextBox is also compatible with management software such as YRCache, LMCache, and Mooncake, providing shared and scalable context infrastructure for long-context, multi-turn dialogue, and Agent inference.
04.From Training to Inference AI Storage Must Also Account for Efficiency and Cost
As one of the earliest domestic vendors to build full-stack AI storage capabilities covering the entire AI process, YanRong has formed storage solutions covering AI training and inference. Represented by the F9000X, YRCache inference storage system, and YRCache ContextBox appliance, these solutions cover key links such as training data reading, model state preservation, and KV Cache management.
This layout corresponds to different stages of AI applications: during training, the F9000X improves the read/write efficiency of data and checkpoints, reducing GPU waiting times; during inference, YRCache and YRCache ContextBox manage, share, and reuse KV Caches to reduce redundant computations. For enterprises, the value of this solution lies not only in increasing bandwidth or reducing latency but also in enabling existing computing infrastructure to handle more training tasks and inference requests. With improved storage efficiency, enterprises no longer need to rely solely on adding GPUs or storage nodes to expand AI service capabilities.
Beyond performance, cost is also a critical consideration in AI infrastructure construction. YanRong Technology states that its hardware cost per unit of performance is reduced by approximately 35% compared to the industry market average, while token efficiency is improved by 120%. While enhancing AI storage efficiency, it enables better control over infrastructure investments. From training data supply to model state preservation and inference context reuse, YanRong aims to use a full-process storage solution to reduce computing power waiting times and resource waste, allowing the GPUs, network, and storage equipment already invested by enterprises to deliver greater value.
05.Conclusion: From High Throughput to High Reuse AI Storage Enters Full-Process Competition
As AI competition enters the stage of large-scale application, storage is no longer just a backend data foundation but has become critical infrastructure affecting GPU utilization, inference efficiency, and construction costs. Through its full-stack AI storage solution covering training and inference scenarios, YanRong helps enterprises make more sufficient use of computing power and infrastructure investments. As model sizes, context lengths, and concurrent requests continue to increase, the competition in AI storage will shift from a single peak performance to the coordinated management of different data, different tiers, and different usage cycles. The construction of AI infrastructure increasingly requires handling more training tasks and inference requests at a more controllable cost.