GPU cluster inference hits a performance bottleneck; brain-inspired chips step in to "turbocharge" it!
Just now, the domestic brain-inspired super node has reached new heights, offering a new solution for cost reduction and efficiency improvement in domestic computing power. Zhidongxi reported on November 4 that at the 2026 World Internet Conference held from November 2 to November 5, the "Internet Light" Expo unveiled ten premier debut achievements.
The "Brain-inspired Super Node for High-Performance LLM Inference FFN Acceleration", jointly applied for by Beijing Lynxi Technology Co., Ltd., China Mobile (Suzhou) Software Technology Co., Ltd., and China Electronics Technology Nanhu Research Institute, was successfully selected. The uniqueness of this solution lies in combining GPUs with brain-inspired chips, where GPUs handle Attention mechanism calculations, while brain-inspired chips specifically undertake FFN (Feed-Forward Network) and MoE (Mixture of Experts) expert layer inference tasks. As early as September, during the 2026 China Computing Power Conference, this domestic GPU + brain-inspired chip heterogeneous hybrid inference system made its global public debut.
From the prototype launch at the Computing Power Conference to the standardized computing unit display at the Internet Light Expo, the domestic AFD (Attention-FFN Disaggregation) route has begun entering the industrial positioning phase. It targets the emerging AFD route in global large model inference architectures. NVIDIA has already combined Rubin GPUs with Groq LPUs to explore next-generation heterogeneous inference architectures. Domestically, Lynxi Technology relies on its self-developed brain-inspired chips to forge a technical implementation path with Chinese characteristics.
01.GPU Inference Efficiency Bottlenecks Hidden in the Division of Labor Between Attention and FFN
Why is large model inference becoming increasingly expensive? The underlying reason points to resource mismatch, as different computational tasks within a GPU have varying demands for hardware resources. Taking the Transformer architecture as an example, each layer contains two core modules: Attention and FFN. The former primarily processes contextual information, requiring frequent reading of KV Cache, thus placing high demands on VRAM capacity and bandwidth; the latter mainly loads model weights for matrix multiplication, undertaking substantial computational tasks.
Current mainstream GPU-based computing architecture chips face two shortcomings when addressing high-performance large model inference needs.
First, the memory wall formed by the von Neumann architecture results in computing capabilities far exceeding data supply capabilities, leading to insufficient utilization rates and power waste;
Second, the dynamic scheduling mechanism of GPUs, while enabling parallel computation, introduces latency uncertainty, often resulting in high latency. When GPUs face high-performance large model inference, Attention relies more on VRAM bandwidth, while FFN requires ample low-latency computing resources. Cramming both tasks into a single GPU makes resource mismatch inevitable.
This issue becomes more pronounced in MoE models. Although the model possesses numerous experts, each Token only activates a subset of them. Computational tasks are dispersed across different experts, making it difficult for FFN to form sufficiently large computation batches, potentially leaving some GPU computing power idle.
How can different modules acquire the most suitable hardware resources? Large model inference architectures are evolving from P/D (Prefill/Decode) disaggregation towards AFD, i.e., module-level disaggregation of Attention and FFN. The former assigns prefill and decode phases to different computing resources, while the latter further separates Attention and FFN within each Transformer layer for deployment. This has become a direction explored by the global industry.
In December 2025, NVIDIA integrated Groq and its compute-in-memory LPU technology into its ecosystem; in March 2026, it first released the Rubin GPU and Groq LPU hybrid cluster solution, expected to be fully launched by the end of this year. Each Groq LP30 features 500MB of compiler-managed on-chip SRAM, striving to keep weights and activations on-chip. Domestically, Lynxi Technology has long been deeply engaged in brain-inspired chip design and large model inference applications. Its KA200 adopts an on-chip compute-in-memory and event-driven architecture, with technical approaches highly similar to Groq LPUs, and the actual proposal and R&D initiation time was no later than NVIDIA's.
02.Handing FFN to Brain-Inspired Chips: How Is Domestic Heterogeneous Inference Achieved?
The core change in this system is allowing GPUs and brain-inspired chips to handle tasks they excel at respectively. In Lynxi's solution, GPUs handle Prefill, KV Cache construction and maintenance, and Attention calculations, leveraging their strong parallel computing capabilities, large HBM VRAM, and mature software ecosystem; brain-inspired super nodes specifically undertake FFN feed-forward networks and sparse MoE expert layer inference, utilizing KA200's compute-in-memory, event-driven, and high on-chip bandwidth characteristics. The KA200 brain-inspired chip uses a 12nm process, with its core advantage lying in on-chip SRAM compute-in-memory and event-driven mechanisms. Traditional GPUs require repeated data movement between VRAM and computing units, whereas KA200 brain-inspired chips keep weights resident directly alongside computing units, eliminating long-distance data transport; the event-driven mechanism schedules computing resources based on input events, reducing unnecessary work; static compilation technology pre-plans instruction and data paths, achieving zero runtime scheduling and deterministic latency.
However, connecting these two types of chips is far more complex than simply assigning computational tasks to different devices. In terms of engineering implementation, Lynxi employs a system composed of 3 to 6 GPU servers and 3 brain-inspired super nodes, with each brain-inspired super node interconnected by thousands of KA200 brain-inspired chips. FFN operators originally on the GPU side are offloaded and deployed on the brain-inspired side; after completing calculations, the brain-inspired side writes outputs back to the GPU side to interface with the model's Attention-related computations. The complete inference process is jointly driven by GPU and brain-inspired nodes, with the brain-inspired side serving as FFN service nodes.
Notably, the FFN part directly utilizes static compilation methods, deploying data deterministically onto the on-chip SRAM of various chips within the super node after compilation, ensuring deterministic calculation and optimization of data pipelines and latency, gradually eliminating latency instability and random jitter. Supporting this synergy are self-developed model compilers, high-speed interconnect protocols, and unified heterogeneous inference engines. The super node hardware is built using HP640 series brain-inspired computing boards and LynxLink high-speed switches, while the software integrates a heterogeneous scheduling engine and unified memory address compilation technology. The system natively supports mainstream large model development frameworks such as PyTorch, DeepSpeed, and Megatron-LM, as well as various mainstream inference engines, requiring no complex secondary modifications to the models.
FFN acceleration capabilities are packaged as independently deliverable computing units, featuring hardware-agnostic and model-agnostic properties, allowing direct mixed networking with existing GPU servers and adapting to air-cooled deployment conditions in current intelligent computing center racks. Compared with NVIDIA's Rubin GPU + Groq LPU solution, Lynxi is following a domestic heterogeneous inference path. It does not require GPUs to exit the inference system but instead adds specialized brain-inspired computing nodes, effectively installing a "turbocharger" for GPUs.
03.Beyond Tens of Thousands of Tokens Per Second: Greater Imagination Lies in Revitalizing Existing GPUs
The viability of architectural innovation ultimately depends on actual output. Previously, the Lynxi team completed heterogeneous inference tests based on the DeepSeek V4 Flash model. Under long context conditions of no less than 32K, pairing 3 to 6 A800 GPU servers with 3 brain-inspired super nodes achieved system output throughput reaching tens of thousands to one hundred thousand Tokens/s. According to comparisons of equivalent investment costs in project materials, the GPU + brain-inspired super node solution improved inference throughput and energy efficiency by over 1 times, while comprehensive business operating costs decreased by more than 40%. This solution has also completed commercial verification of mainstream large models in the intelligent computing centers of leading telecom operators.
More noteworthy is that current test results are still advancing toward higher levels. According to the project team, DeepSeek V4 Flash adopts a MoE architecture, where each Token only activates 6 out of 256 routed experts. Subsequent optimizations through mixture-of-experts tuning and model quantization stacking are expected to further reduce hardware resource requirements. After stacking these two optimizations, the project team aims to increase system output throughput to hundreds of thousands of Tokens/s. From an industry perspective, the value of this solution also lies in leveraging existing assets.
Currently, many intelligent computing centers have deployed large numbers of GPU servers, with some existing GPUs lagging behind newer high-end products in performance. Lynxi has already completed integration with earlier models of Iluvatar CoreX GPUs, achieving over 1 times improvement in both inference output and energy efficiency. In the future, FFN brain-inspired super nodes are expected to connect different types of GPUs, forming more flexible heterogeneous computing clusters. For domestic GPU manufacturers and intelligent computing center operators, this provides a new path for expansion and cost reduction.
04.Conclusion: Installing Turbochargers for GPUs, Token Factories Welcome a New Foundation
From process node competition among single chips to system-level collaboration between GPUs and brain-inspired chips, domestic computing power is exploring new avenues for performance growth. As large model inference enters the stage of scaled application, factors determining computing power competitiveness have extended to hardware division of labor, data movement, cross-architecture scheduling, and system energy efficiency.
Lynxi 's FFN-accelerated brain-inspired super node being selected as one of the ten premier debut achievements at the Internet Conference signifies that the domestic heterogeneous inference route is moving from technical validation towards standardized product exploration. Allowing chips of different architectures to perform their respective duties and maximizing the value of existing computing power may become a key variable in the next phase of domestic AI Infra competition. This simultaneously implies that the dependence of Token factory computing solutions on pure GPU architectures and advanced HBM will decrease, ushering in an era of hybrid heterogeneous clusters where brain-inspired chips "turbocharge" GPUs.