Author | Liu Yu Editor | Chen Junda In recent years, AI infrastructure construction has primarily relied on expanding computing power scale to meet the growing demands of models. However, as the parameter scale of large models continues to increase and applications such as AI agents drive rapid growth in inference demand, computing costs, energy consumption, and resource utilization efficiency have gradually become new bottlenecks for industrial development. Data from the National Data Administration shows that by the end of March this year, China's daily average Token invocation volume exceeded 140 trillion, an increase of over a thousand times compared to the beginning of 2024.
Facing the continuously growing inference workload, the industry is exploring various paths to improve computing efficiency, including algorithm optimization, software optimization, and novel computing architectures. Among these, sparse computing has emerged as one of the closely watched technical directions. By identifying redundant computations in models, this technology reduces invalid computations and improves computing resource utilization efficiency while controlling the impact on accuracy, thereby lowering the computational consumption during large model inference. However, transitioning from technical research to large-scale application, sparse computing still needs to address the coordination issues among algorithms, software, chip architectures, and application ecosystems.
To break through the energy efficiency bottleneck under the traditional computing paradigm and lower the threshold for industrial applications, domestic AI chip startup Moffett AI (hereinafter referred to as "Moffett AI") initiated the establishment of the Sparse Computing Industry-University-Research Alliance, uniting universities, research institutions, and industry partners. This initiative aims to promote the transition of sparse computing from single-point technical exploration to industrial collaboration. For Moffett AI, this also signifies that its development around the sparse computing technology roadmap is further extending from chip products and computing power deployment to industrial ecosystem construction.
01.Establishing the Sparse Computing Industry-University-Research Alliance to Promote a Collaborative Closed Loop of Algorithms, Software, Chips, and Applications
Current mainstream AI computing systems are primarily built on the dense computing model, which performs calculations based on complete matrices. However, as model scales continue to expand, a massive amount of computing resources is consumed on processing low-value information. How to reduce invalid computation has become a crucial direction for improving the efficiency of AI infrastructure. Sparse computing is a technical path developed precisely in this context. In AI neural networks, the vast majority of weights and activation values are zero or near-zero invalid values. These values contribute minimally to the final result yet occupy a large amount of computing resources.
The core idea of sparse computing is to skip the multiplication and addition operations of these invalid values, performing calculations only on non-zero values (valid values), thereby reducing the computational workload and data movement pressure. Compared to the inherent shortcomings of traditional dense computing, sparse computing offers the advantages of high efficiency, energy saving, cost reduction, and efficiency enhancement, making it one of the breakthroughs to resolve the current AI computing power energy efficiency bottleneck. However, there is currently no complete sparse computing industry chain ecosystem in China. Meanwhile, developing sparse computing from technical principles to large-scale industrialization still faces significant challenges. On the one hand, the sparse patterns at the model level need to match the hardware architecture.
Different models and tasks have varying requirements for sparse structures, necessitating collaborative optimization between algorithm research and chip design. On the other hand, the existing AI software ecosystem is primarily built around traditional computing models. Sparse computing requires adaptation by compilers, frameworks, and toolchains to reduce development and deployment costs. Therefore, the industrialization of sparse computing is not simply about improving the performance of a single chip, but rather a systematic engineering project involving algorithm innovation, software optimization, hardware design, and application verification. Such complex coordination requirements mean that the large-scale implementation of sparse computing is difficult to achieve by a single enterprise fighting alone. Joint participation and division of labor across the upstream and downstream of the industry represent a more realistic path. The initiation of the Sparse Computing Industry-University-Research Alliance by Moffett AI is precisely aimed at addressing this industrial coordination need and promoting connections among different segments.
Following its establishment, the Alliance will address issues such as how to adapt different models, how to improve software toolchains, how to optimize chip architectures, and how to deploy industry applications at scale through joint research, technical evaluation, standard formulation, and application promotion. This will drive deep collaboration across the upstream and downstream of the sparse computing industry chain. Its goal is not to replace the existing AI computing system, but to provide a new efficiency optimization path for AI infrastructure against the backdrop of growing large model inference demands. Currently, the Alliance has gathered multiple forces from universities, research institutions, and industry partners, and the list of relevant partners is continuously being updated and expanded.
From the perspective of industrial structure, Alliance members cover multiple segments including basic research, technology R&D, and application scenarios, which helps promote the transition of sparse computing from laboratory research to engineering practice. In terms of the technology roadmap, the Alliance plans to build a full-stack sparse technology roadmap, advancing full-stack sparse technology R&D based on the "6S Technology Architecture." Herein, Sparse represents identifying and utilizing sparsity, Speed represents enhancing computing throughput, Swift emphasizes increasing computing power density, Saving focuses on reducing energy consumption, Stable emphasizes system stability, and Smart focuses on intelligent optimization and adaptive computing.
Specifically, in the direction of high-ratio semi-structured and mixed sparsity, the Alliance will explore automatic pattern search and accuracy recovery for N:M sparsity, block sparsity, and fine-grained sparsity. In the direction of generalized sparse large models, it will expand into two-layer sparsity, Token filtering, Sparse Attention, and multimodal sparsity. In the direction of fusion optimization of sparsity and low-bit quantization, the Alliance will promote joint optimization of sparsity with INT8/INT4/FP8/FP4 and mixed-precision search. In the direction of next-generation sparse chip architectures, it will tackle direct computation in the compressed domain, sparse metadata engines, and dynamic sparse scheduling. In the direction of dynamic-static collaborative sparse computing, it will achieve joint compilation and adaptive switching of static structural sparsity and dynamic activation sparsity. In the direction of SparseBench and standard ecosystems, it will establish a five-level indicator system and industry standards covering TTFT, TPOT, P99, energy consumption, and per-Token cost. This tackling roadmap will be implemented in phases according to the established plan.
The Alliance plans to complete sparse-aware modeling to build the underlying algorithmic foundation in early 2027, and achieve multi-mode hybrid sparse fusion and single-machine energy efficiency optimization by August of the same year. In October 2028, the Alliance will break through the MoE communication bottleneck and implement sparse deployment capabilities for thousand-card computing clusters. By the end of 2029, the Sparse Computing Industry-University-Research Alliance will complete the full-stack sparse collaborative layout of training, algorithms, chips, and interconnected software and hardware, finalize the overall native sparse solution, and form industry standards, achieving a complete closed loop from algorithm R&D to industrial standardized implementation of sparse computing.
02.Commercial Projects Step Up Token Cost Reduction, Opening Industrial Practices to Create Reproducible Computing Efficiency Enhancement Solutions
The planning and implementation of the full-stack roadmap for the Sparse Computing Industry-University-Research Alliance is not castles in the air. If the establishment of the Alliance addresses the issue of collaborative innovation in the industrialization of sparse computing, then the actual business implementation determines whether this technology roadmap can truly respond to the demand for improving AI infrastructure efficiency. Under the current Token economic system, the pain points of large model commercialization lie in "high concurrency, high real-time requirements, and persistently high per-Token costs," with multiple factors jointly driving up overall inference expenditures.
For cloud service providers, industry customers, and intelligent computing centers, as the scale of Token invocations grows, the demand for computing power is no longer just about simply increasing the number of GPUs, but requires improving unit computing output under limited hardware resources. Against this backdrop, sparse computing can optimize the computing process, increase the volume of effective tasks completed per unit of computing power, and reduce the computational cost during large model inference. However, different application scenarios have varying requirements for inference latency, throughput, energy consumption, and costs. Only by embedding sparse computing capabilities into actual computing power deployment environments can its advantages in improving Token efficiency be fully realized.
Moffett AI is one of the few domestic startups pursuing a differentiated sparse computing route. Its promotion of multi-regional computing power deployment aims to leverage the practical value of sparse computing in improving inference efficiency across different business workloads. For example, the thousand-card inference cluster deployed by Moffett AI in the Northwest region primarily targets the intelligent transformation needs of traditional industries, focusing on real-time AI businesses such as factory security and machine vision inspection in electronic manufacturing and consumer goods processing scenarios.
Industrial visual quality inspection and factory security duty require uninterrupted processing of massive images and video streams, imposing strict constraints on inference latency. Such industrial scenarios demand a stable supply of large-scale computing power 24/7. The computing power utilization rate and per-task computing cost determine the return on investment for factory intelligentization projects. By relying on the sparse computing architecture to optimize the inference link, redundant and repetitive computations can be reduced, enhancing the business carrying efficiency of a single computing power node, thereby helping enterprises lower the landing cost of industrial AI. In the Southwest region, Moffett AI combines local energy conditions to build a low-power green computing power pool. As AI inference demand continues to grow, energy consumption has become a significant constraint on data center operations.
For inference services that require long-term operation, increasing the effective computing volume generated per unit of electricity holds more long-term value than simply expanding the computing power scale. By reducing invalid computing volume and improving energy efficiency, sparse computing helps lower the energy costs during the operation of inference services. In the East China region, its related computing power deployment focuses on computing-intensive scenarios such as bioinformatics and medical health, supporting tasks like gene sequencing analysis and protein structure prediction through AI computing power. In the North China region, it explores multimodal visual applications such as facial recognition and posture recognition around urban governance and community intelligentization.
If every inference task relies on traditional dense computing methods, computing costs will continuously increase as data scales grow. By improving inference efficiency and optimizing the computing process, sparse computing can help such computing-intensive applications reduce computing power consumption, while increasing the number of video analysis tasks carried per unit of computing power, enabling AI capabilities to cover more terminals and application nodes. Although these application scenarios span different industries, from an industry trend perspective, as more and more enterprises embed large model capabilities into their business processes, the future focus of AI infrastructure competition may lie in how to reduce enterprises' long-term operating costs through architecture optimization, software adaptation, and computing efficiency improvement. The role played by sparse computing in the current Token ecosystem will also become increasingly important.
In addition to existing commercial deployments, Moffett AI is also advancing the new-generation sparse-dedicated inference chip SparsePrime for future large model inference demands. Designed based on the self-developed Antoum2.0 chip architecture, this product is positioned for large model inference scenarios in intelligent computing centers and data centers. It improves sparse computing efficiency through collaborative software and hardware optimization and is expected to enter mass production this year.
Compared to previous generation products, SparsePrime focuses on enhancing adaptability for general inference scenarios. It relies on toolchain optimization to simplify the model migration process, is compatible with mainstream deep learning frameworks and inference supporting ecosystems, and helps developers implement sparse computing at lower deployment costs. Moffett AI's SparsePrime product and its existing computing power deployment practices have verified the possibility of sparse computing technology in improving inference efficiency under real business workloads, providing a practical foundation for subsequent industrial ecosystem construction.
By establishing this Alliance, Moffett AI also aims to open its existing technology paths and industrial practice experiences to more universities, institutions, and enterprises, thereby promoting the formation of reproducible commercialization solutions for computing power cost reduction and efficiency enhancement in sparse computing. Meanwhile, Moffett AI has completed a nearly 1 billion RMB Series C funding round. This round of financing brings together top state-owned venture capital institution Shenzhen Capital Group, as well as industrial capital and market-oriented institutions such as Yanshan Technology, Greater Bay Area Common Home, Liding Capital, and Yunsheng Capital. The diversified shareholder structure reflects, to a certain extent, the recognition from both the industrial side and the capital market of the sparse computing technology roadmap and ecosystem layout.
03.Supplementing Interdisciplinary Talents in Sparse Computing to Improve the Integrated Construction of the Domestic AI Computing Power Ecosystem
In recent years, the domestic AI chip industry has focused more on hardware product capability building, including chip design, performance optimization, and application adaptation, but the overall ecosystem's software and hardware coordination has been relatively weak. The development of sparse computing naturally requires deep coordination among algorithms, software, and chips, which also creates an opportunity to promote the overall improvement of the domestic AI computing ecosystem. Universities and research institutions possess research capabilities in algorithms, models, and computing theories, while industrial enterprises are closer to chip R&D, software optimization, and actual application scenarios.
Through the industry-university-research collaboration in sparse computing, research directions can be brought closer to industrial needs, while problems in industrial practice can be fed back to the technical research stage, driving continuous technological optimization. Since sparse computing involves multiple fields such as algorithm research, chip architecture design, and system deployment, the establishment of the industry-university-research alliance can further strengthen the connection between university research and industrial practice, cultivating interdisciplinary talents who understand both AI model requirements and underlying computing architectures for related fields, thereby consolidating the continuous innovation capability of the domestic sparse computing ecosystem. In addition, the rapid growth of AI computing power scale has brought pressure on energy consumption and operating costs. As the construction of intelligent computing centers continues to advance, future computing power infrastructure will not only need to enhance computing capabilities but also focus on energy efficiency levels and long-term operating costs.
This trend echoes the direction of national computing power infrastructure construction. The 15th Five-Year Plan continues to emphasize the development of core digital economy industries. As the scale of the digital industry expands, the demand for AI computing power will further increase. During the promotion of the "East Data, West Computing" project, requirements for the green and low-carbon development of data centers have also been raised, with the proportion of green electricity used in newly built data centers at national hub nodes further increasing on the basis of 80%. During this year's Two Sessions, "computing-power and electricity coordination" has also become an important direction for new infrastructure construction, further emphasizing the coordinated matching between computing power layout and energy supply. The path of carrying out industry-university-research cooperation around sparse computing aligns with the national macro strategy, helping to improve the utilization efficiency of computing power resources, promote "computing-power and electricity coordination," and reduce the total cost of ownership of infrastructure, thereby alleviating the contradiction between the rapid growth of computing power and energy supply and low-carbon development.
04.Conclusion: Lowering Computing Power Energy Consumption Costs and Broadening the Technical Path for Domestic Computing Power Infrastructure
According to calculations by Zhongtai Securities, the future scale of AI inference demand is expected to reach 5 to 10 times that of the training stage. Compared to the training stage, which focuses more on centralized computing capabilities, the inference stage places greater emphasis on unit cost, response speed, and deployment efficiency. How to improve unit computing power output and reduce the operating costs of AI applications is becoming an important topic for the next stage of computing power infrastructure. Against the industry backdrop where Moore's Law is approaching its bottleneck and AI computing power energy consumption and operating costs continue to rise, efficient computing technologies represented by sparse computing are breaking the inherent paradigm of traditional dense computing, providing a technical path for the AI computing power industry to transform from "scale stacking" to "efficiency and quality improvement."
In the long run, as the collaborative mechanism of the Sparse Computing Industry-University-Research Alliance continues to be implemented, technical standards are gradually improved, and full-stack sparse technology continuously iterates and matures, sparse computing may further penetrate core scenarios such as large model inference, multimodal applications, and industry computing power clusters. It will step up the development of domestic new AI infrastructure and further improve the local AI ecosystem chain.