What is the hottest keyword in the field of AI computing in 2026?
The answer is obviously the supernode.
Almost all computing power vendors are deploying around supernodes. A consensus has been formed: the era of competing on single-card computing power is over, and system-level synergy is the key to victory.
Adopting a Scale-up tightly coupled approach, supernodes integrate a large number of AI chips at high density to build a unified high-speed computing domain, which is indeed an inevitable choice for the development of computing power.
The entire industry is fiercely competing on supernodes, but the direction of this competition is undergoing subtle changes.
On September 23, Inferact, an overseas TPU inference optimization team, open-sourced a set of tpu-megakernels (mega-operators for TPUs), using 16 Google TPU v7 chips to fit the entire Kimi K3 into a single kernel. Coupled with DSpark speculative decoding, the low-concurrency Decode throughput reached 709 tokens/s.
In the past, the competition focused on "scale"—the number of supernode chips, VRAM capacity, and the number of model parameters that could be hosted were the core metrics for vendors to compete against each other.
Today, the focus of the industry's evaluation of AI computing power has shifted to "speed"—everyone realizes that supernodes cannot be judged solely by the scale of stacked cards, but more importantly, by Token speed.
Domestic vendors are also actively deploying. At the 2026 AI Computing Conference not long ago, IEIT SYSTEMS unveiled its latest supernode—the Meta Brain SD200 Ultra. At the launch event, several figures they presented attracted the industry's attention:
"The SD200 Ultra tightly couples 128 domestic AI chips in a single machine, supports models with up to 10 trillion parameters per machine, can natively run the 2.8 trillion parameter Kimi K3 large model, and achieves a single Token generation latency of under 5.85 milliseconds for the first time."
5.85 milliseconds, equivalent to 170 Tokens/s for a single user, what level does this represent?
To put it this way, although there is no third-party evaluation in the industry yet, generally speaking, the single-user Token output speed of many domestic supernodes without optimization is often around 20 Tokens/s. For the API interfaces of various large models, the Token generation speed is 60 Tokens/s, while the international leading level performance can reach 200 Tokens/s (with latency under 5 milliseconds).
The SD200 Ultra is already at the top level of domestic supernodes, about five times the average value.
Then the question arises: why does IEIT SYSTEMS emphasize Token output efficiency so much? What considerations are behind this unconventional approach? And how do they achieve such a fast output speed?
Next, Xiao Zao Jun will provide an in-depth interpretation.
█ Computing Power Competition Enters a New Stage, Agent Opens the Era of Token Efficiency
The core reason IEIT SYSTEMS focuses on Token output efficiency lies in its judgment of the development trend of AI computing power.
Looking back at the competition in AI computing power, it can actually be roughly summarized into three stages:
First is the earliest stage of single-chip performance competition. In this stage, the industry focused on researching how to improve single-chip computing power, VRAM bandwidth, and capacity. Single-chip performance is an important foundation for system performance.
Later, as model scales continued to expand, single-card and single-machine setups could no longer meet the demand. The industry began to research multi-card and multi-machine collaboration to build larger-scale computing card clusters. Supernodes belong to the tightly coupled computing domain, which is equivalent to "condensing" more cards into a single computer, offering stronger performance.
In this stage, the industry focused on scale, which means competing on the number of chips in supernodes.
However, in the pursuit of scale, it has been found that data exchange, synchronization, and memory access between chips are becoming increasingly complex. Issues such as communication overhead, data movement, and computation waiting are also becoming increasingly thorny.
It is difficult for the overall computing power of the system to grow linearly with the number of computing cards, and the performance loss is increasing. System-level optimizations such as interconnects and operators have also become important factors affecting effective computing power.
While the supply side of computing power encounters bottlenecks, the demand side has also changed.
The AI wave has transitioned from the generative AI era to the agentic AI era, with Agents becoming the mainstream business form. User interaction has shifted from one-time Q&A to continuous autonomous task execution.
Agents are typical long-chain businesses. They break down goals, generate plans, call tools, read return results, and then iterate through reasoning, engaging in such multi-round loops and continuous thinking.
In the multi-round loops of Agents, every tool call and result reading is accompanied by the generation and consumption of Tokens. If the word generation speed (Tokens/s) remains at the "typewriter" level of twenty or thirty, not only will long-chain thinking completely stall the business process, but the extremely low Model Flops Utilization (MFU) will also directly break through the commercial profitability bottom line of large model vendors.
To put it bluntly, a large amount of computing power is wasted on waiting for Token output, repeated data movement, and communication blockages. No matter how high the peak hardware computing power is, it cannot be transformed into implementable and profitable intelligent productivity.
Based on the above background, IEIT SYSTEMS proposed the unique Capability & Capacity computing power supply concept.
They pointed out that AI computing in the Agent era is forming two equally important development directions:
Capability AI Compute, which supports breaking through the upper limit of intelligence, allowing AI to help humans explore the unknown and solve problems that could not be solved in the past.
Capacity AI Compute, which achieves sufficient supply of intelligence, making high-level intelligence increasingly abundant, more economical, and easier to access.
Simply put, Capability determines how high AI can fly, and Capacity determines how far AI can fly. The former is the "sharp knife" for breaking through the upper limit, and the latter is the "cornerstone" for universal implementation.
From the perspective of Capability, supernode products launched to the market must have sufficiently strong capabilities to meet users' needs for running cutting-edge large models, applied in challenging scenarios such as scientific research innovation, business decision-making, and data processing.
Supernodes have been in the public eye for quite a long time, but previously when vendors in the circle promoted supernodes, intentionally or unintentionally, they would all take scale as the key point, but rarely mentioned speed. Because, as mentioned earlier, scale expansion will inevitably bring efficiency bottlenecks.
IEIT SYSTEMS puts speed at the highest priority, which truly avoids the virtual and focuses on the practical, pointing out the key to the problem—a good supernode product is not judged by how many chips it can connect, but by whether the newly added chips can continuously contribute effective computing power.
Maximizing the computing scaling efficiency is more meaningful than blindly "stacking cards". Only by making these chips form a tighter whole can extreme efficiency be exerted, ensuring that every investment delivers value.
The entire design of the Meta Brain SD200 Ultra revolves around this concept.
Compared to last year's SD200, the SD200 Ultra first upgrades the chips, expanding from 64 cores to 128 cores. The computing power is directly doubled, and the VRAM is also directly doubled. A single machine can support up to 8TB of GPU VRAM, capable of running leading open-source models like Kimi K3 (2.8 trillion parameters). In the next year or two, top large models will enter the 10-trillion-parameter era, and the SD200 Ultra is fully capable of handling it.
The SD200 Ultra brings Token output efficiency far exceeding the industry average, providing users with fast-responding intelligent supply, allowing users to confidently embrace Agent applications.
This powerful Capability AI also lays the foundation for the universal implementation of Capacity AI.
Liu Jun, Chief AI Strategy Officer at IEIT SYSTEMS, put it very straightforwardly: "Today's cutting-edge capabilities are tomorrow's universal foundation."
█ Architectural Innovation + Super Operators, Building an Ultra-Fast Intelligent Computing Foundation
Next, let's look from a technical perspective at how the SD200 Ultra achieves high Token output speed.
Let's first look at what the main factors affecting the Token output rate are.
Take the world's largest open-source model, Kimi K3, as an example.
This model has a total of 2.8 trillion parameters, 104B active parameters, and weight files of about 1.56TB. For ordinary AI servers, the GPU VRAM cannot even hold the weights, let alone a million-Token context. Therefore, the official recommendation is to scale up to 64 cards or even larger to accommodate it.
Even if it can be accommodated, the requirements for the VRAM bandwidth of the computing system and inter-node communication are extremely high.
According to SemiAnalysis, for every forward computation executed by K3, the HBM bandwidth demand alone is as high as about 1.5TB. 896 experts need to be distributed across multiple cards, and each forward propagation triggers over 120 All-to-All communications.
This means that the larger the scale, if the interconnect bandwidth and latency cannot keep up, a large amount of computing power will be wasted on waiting for data movement between multiple cards.
Generally speaking, an unoptimized supernode running Kimi K3 may only have an initial performance of around 10 tokens/s.
Large model inference usually goes through two stages: Prefill and Decode.
Prefill centrally processes inputs, is compute-intensive with chunked data, and communication overhead can be amortized by a large amount of computation.
Decode is different. In the decoding stage, output is generated Token by Token. The computation for each step is limited, and the fixed overhead of communication and synchronization cannot be amortized. Consequently, the proportion of communication time in the single Token generation time significantly increases.
Therefore, decoding efficiency directly determines the efficiency of the entire inference system.
The larger the cluster scale, the longer the communication links and the more nodes participating in synchronization. In the decoding stage, with fine computation granularity and high communication frequency, collective communication and cross-node synchronization will continuously amplify communication overhead.
When communication time, tail latency, and scheduling costs gradually offset the benefits of newly added computing power, the marginal improvement in Token throughput will slow down—the scale is expanding, but efficiency may not grow synchronously.
Therefore, we can see that to improve Token output efficiency, the key lies in breaking through core bottlenecks, significantly enhancing communication link capabilities, shortening communication time, eliminating tail latency, reducing scheduling costs, and thereby improving decoding efficiency.
The technological innovation solutions adopted by IEIT SYSTEMS mainly include two aspects:
First, the proprietary Co-Fabric tightly coupled architecture, reshaping the chip interconnect foundation.
Supernodes, to put it simply, hinge on the interconnect technology between chips. NVIDIA's NVL72 and Huawei's Ascend 950—the former achieves high-bandwidth intra-domain interconnect through NVLink, an innovation based on PCIe. The latter relies on the self-developed UnifiedBus to build a tightly coupled topology, an improvement based on Ethernet.
Both NVLink and UnifiedBus are targeted at specific chips. As a vendor like IEIT SYSTEMS, catering to multiple types of computing power is an inevitable choice, so they must take a more universal and open interconnect route.
The solution they provide is to create a new architecture that includes a hardware-software collaborative technical system such as interconnect protocols, system architecture, underlying software, and communication libraries. This architecture is Co-Fabric.
Co-Fabric is a deeply improved bus protocol, referencing the PCIe physical layer foundation. It extends the computing domain bus paradigm from the xPU domain to cover all system resources (compute, memory, network, and management), incorporating them into a single interconnect network.
Co-Fabric Architecture
By providing unified addressing across host domains, native P2P communication with Load/Store semantics, and a streamlined protocol stack, Co-Fabric builds a unified data movement plane, breaking host domain boundaries and maximizing end-to-end data movement efficiency across the entire supernode.
The first feature of Co-Fabric is unified memory access.
Co-Fabric inherits the core advantages of the bus architecture—native memory semantics. Through the pioneering "shadow device automatic enumeration" mechanism, it builds a globally unified flat address space for the entire supernode.
Regardless of which server node these AI chips are physically distributed on, for the system and developers, they are uniformly mapped as locally visible virtual devices. The chips can directly use the lowest-level Load/Store instructions to read and write remote VRAM across domains, completely eliminating the tedious VRAM registration and multi-layer address translation overhead of traditional Ethernet.
Unified Addressing
The second feature of Co-Fabric is the streamlined protocol.
To maintain ultra-high speed during cross-domain scaling, Co-Fabric has undergone "extreme streamlining" of network protocols. Addressing the issue where the heavy transport and network layers of traditional Ethernet affect transmission speed and data integrity, Co-Fabric reduces unnecessary protocol processing steps through a four-layer streamlined protocol stack (Physical, Link, Fabric, and Semantic), improving data transmission efficiency and end-to-end communication performance.
Its Fabric layer uses port identifiers embedded in packet headers for hardware-level routing. Data packets can be directly delivered across domains at the switch hardware level, completely bypassing the host CPU and network protocol stack.
Four-Layer Streamlined Protocol Stack
More importantly, Co-Fabric natively supports Credit-based flow control and retransmission mechanisms at the link layer, compressing the latency of reliable transmission between nodes to the nanosecond level.
The third feature of Co-Fabric is the open architecture.
Co-Fabric is highly open. It incorporates compute, memory, and network resources into the same interconnect domain, supporting the elastic scaling of supernodes from dozens of cards to thousands of cards.
The Meta Brain SD200 Ultra is the perfect product implementation based on Co-Fabric innovation.
It adopts a 3D Hyper Mesh architecture, incorporating 128-core AI chips into the same high-speed interconnect domain, building a native memory semantic communication network. The All-to-All communication latency is as low as 0.69 microseconds, significantly reducing data interaction overhead between chips, allowing hundreds of chips to work collaboratively as a single chip, and building the underlying hardware channel for high-speed Token generation.
The SD200 Ultra builds a unified address space through global unified addressing, virtual shadow device registration, and cross-domain address translation. Coupled with symmetric memory peer-to-peer interconnect technology, AI chips can directly read and write remote chip VRAM just like accessing local VRAM, and directly initiate P2P access. This saves a series of overheads such as message packaging, buffer transit, and multi-layer address translation, reducing AllReduce communication time by 3.5 times.
In terms of cost and power consumption, the SD200 Ultra directly abandons expensive Ethernet cards and Ethernet switches, adopting pure hardware Retimers combined with Co-Fabric switch chips and short-distance passive copper cable connections. This drastically reduces the hardware cost of the interconnect network itself by about 80%, and power consumption is also reduced by about 5%.
This design reduces optoelectronic conversion links and lowers the risk of task interruption caused by link failures. It can support the stable operation of long-horizon Agent tasks, achieving an excellent system ROI (Return on Investment).
Second, super operators combined with AI self-optimization achieve full-process Token acceleration.
The Meta Brain SD200 Ultra has also made bold innovations in operator optimization.
Traditional inference frameworks execute a large number of fragmented basic operators serially. Each operator launch brings scheduling overhead and HBM memory access overhead. The launch latency caused by high-frequency switching between operators leaves a large number of wasted "bubbles" on the HBM bandwidth.
The Meta Brain SD200 Ultra builds a super operator agent using Kimi K3, achieving compute-communication fusion and Tile-level pipelined super operators. It fuses numerous basic operators in KDA, Gated MLA, and MoE into large operators that execute compute and communication collaboratively. The number of operators is reduced by 10 times, significantly reducing operator launch times and HBM access overhead, hiding communication latency, eliminating "bubbles", and improving the overall execution efficiency of the GPU.
Through the approach of "optimizing K3 with K3", they achieved a more than 3x improvement in the inference performance of Kimi K3 on the Meta Brain SD200 Ultra, compressing Token generation latency from the compute layer.
The above system-level technological innovations truly transform peak hardware computing power into real, usable effective inference computing power, raising the Model Flops Utilization (MFU), and turning high Token speed from a theoretical metric into a stable and runnable business capability.
For users, what they care about is the experience, and in the Agent era, the most important focus of the experience is the Token generation speed. It directly determines the fluency and naturalness of agent interaction.
IEIT SYSTEMS focuses on user needs, transforming years of technological accumulation into product innovation, providing users with perceptible experience improvements. This direction is obviously correct and worthy of praise!
█ Conclusion
The trend for supernodes has shifted: from competing on the scale of hardware stacking to measuring whether the system can stably, efficiently, and cost-effectively continuously produce Tokens.
In the Agent era, the value of computing power is no longer judged by the paper peak of chips, but by whether it can smoothly complete long-chain intelligent tasks. IEIT SYSTEMS's publication of supernode performance benchmarks is worthy of serious review and reference by the entire industry.
The AI wave continues, and supernode technology will also continue to iterate. We look forward to continuous breakthroughs in supernodes in dimensions such as computing power utilization efficiency, interconnect bandwidth, and energy efficiency ratio. We also look forward to more system-level innovations moving from "laboratory metrics" to "scaled production environments", truly driving humanity to accelerate towards the AGI era.