How wide is the gap between theoretical computing power and actual output? At WAIC 2026, multiple industry insiders provided an unsettling reference figure: less than 30%. In other words, after an AI server is delivered to a user, the computing power that is actually effectively utilized may account for less than 30% of the numbers on the specification sheet.
This is not merely the judgment of a few individuals. Chip companies across different fields have concurrently shifted the topic from "how much computing power is available" to "how well the computing power is utilized." The shift in this narrative itself is a signal: the AI industry has already crossed the first threshold of "whether it can be done" and is advancing towards the stage of "how well it works."
The core contradiction in this stage is migrating upward from the chips themselves.
01 Inference Surpasses Training: Not a Prediction, but an Established Fact
In 2024, hot money in the industry almost entirely flowed towards the training side; by 2026, the computing power demand for inference has already surpassed that for training.
Training chips pursue "learning better," while inference chips pursue "using more efficiently." Fu Qingping, Senior Director of Commercial Products and Solutions at Sunrise, stated that these two different design philosophies correspond to two different business logics. The granularity of inference scheduling is being refined from the task level to the operator level: "The finer the granularity, the higher the efficiency; for every bit of efficiency gained, the token cost drops by a corresponding amount." This means that the competition for inference chips has penetrated deep into the scheduling efficiency at the model layer. A dimension that seemingly belongs to the software realm is becoming a core selling point for chip products.
From the perspective of video processing, the explosion of inference demand is more intuitive. VPU manufacturers predict that 80% of future videos will be generated by AI, and according to estimates, the token consumption for video generation is 1,000 times that of text generation. It is introduced that by offloading video-related computations from GPUs to dedicated chips, VPU+GPU collaboration is expected to reduce video generation costs by 80%.
Yang Dawei, VP of Strategic Ecosystem at Houmo AI, stated: "Training determines the upper limit of a model's capabilities, while inference determines whether AI can be deployed on a large scale." As inference becomes the main axis of the industry, the demand structure of the chip industry changes accordingly. The cloud requires scalable inference chips, video scenarios require VPUs, fast thinking at the edge requires NPUs, and low-power edge inference requires a computing-storage integration architecture. General-purpose GPUs are no longer the only answer; specialization is transforming from theoretical discussion into commercial reality.
02 Computing Power Economics Becomes Industry Consensus
Zhu Zhaoyuan, Co-founder and CEO o fNetint, believes that the fundamental reason for the reduction in computing power costs is the improvement in efficiency. Understanding the iteration of hardware products through the upgrade of transportation modes: from horse-drawn carriages to automobiles and then to trains, the cost of traveling from point A to point B decreases, but the fundamental reason for this reduction is not "saving," but the "increase" in efficiency.
This explains why, at WAIC 2026, multiple companies no longer prioritize TOPS figures, but have started to discuss "comprehensive costs." Data center customers focus on at least six dimensions when evaluating chip solutions: stability, performance, latency, power consumption, image quality, and TCO. "Customers don't care what PU you call it." The fact behind this statement is: when AI moves from the laboratory to the production environment, the evaluation criterion is not your test score, but your work results.
Inference chips reduce costs by trimming training-related redundant functions at the hardware level. This reflects the trend from general-purpose to specialized. From the earliest CPUs to GPUs, and then to DPUs and VPUs, each specialization corresponds to a certain type of computing demand being separated from the general-purpose platform. The case of VPU+GPU collaboration reducing costs has already provided commercial validation for this trend. At this stage, domestic chips choosing the GPU route is a market choice. Hardware manufacturers must face the market, and products must be iterated generation by generation without interruption; the most controllable risk lies in general-purpose GPUs. However, looking further ahead, various computing power chips will develop towards segmented fields, forming unicorns in specific niches.
The cost reduction solution for edge chips relies on algorithm partners to adapt to different scenarios. If the category is large enough (such as smartphones and automobiles), customization costs can be borne; other scenarios are covered by a general-purpose base, allowing the dividends of underlying cost reduction to benefit more industries. From the edge perspective, the problem is more complex. The three rigid demands for edge inference are privacy, cost, and latency in physical world interaction, and these three dimensions determine that the design logic of edge chips differs from that of the cloud. Liu Jianwei, Co-founder and Vice President of Axera, pointed out the logical differences between the edge and the cloud: the cloud prioritizes experience before reducing costs, while the edge prioritizes costs before improving experience. This difference in sequence directly affects the trade-offs in chip design. The business model of edge AI is also changing as a result. When users find that subscription fees might be more expensive than the hardware itself, demands to shift from a subscription model to a one-time buyout begin to emerge.
The AI chip industry is entering a mature stage with TCO as the core consideration. At this stage, the value of software optimization capabilities and system engineering capabilities may exceed the hardware parameters themselves.
Intertwined with this is another divergence: where the next AI revolution will break out. Some believe that revolutions will occur in both the cloud (AI video generation) and the edge (embodied intelligence); some define 2026 as the "first year of physical AI," showing the most optimism for robotics; while others believe that the entire industry will experience an explosion, and the fastest to land will undoubtedly be the industries that understand AI the best.
03 First Year of Physical AI: Where Has Embodied Intelligence Arrived?
Embodied intelligence has been mentioned by multiple companies as the next explosion point. The robotics exhibition area at WAIC provided evidence: unlike previous years' performance-oriented robots that fetched water or boxed, this year there are robots capable of continuously performing delicate tasks for long periods, moving from exhibition booth shows to factory production lines.
However, the industrial form of embodied intelligence is far from converging. Some advocate a broad definition—machines that can work autonomously, such as robot vacuums and lawnmowers, can be called "robots." Others lean towards the humanoid route, reasoning that all human infrastructure is designed for the human form, making humanoid robots the most widely applicable and the most versatile hardware carrier, with costs decreasing after scaling. However, at this stage, the manufacturing cost of humanoids remains high, and it ultimately depends on which form achieves cost feasibility first.
The degree of robots' reliance on AI is fundamentally different from other terminals. Companies pursuing the computing-storage integration route provided a sharp contrast: a PC without AI is still a PC, a car without AI is still a car, but a robot without AI is just a piece of iron. This means that robots have the most urgent rigid demand for edge AI, reflected in three dimensions: privacy (data cannot be uploaded to the cloud), token costs (frequent cloud calls are too expensive), and the real-time performance and safety of physical interaction (cannot lose control when there is no signal in a tunnel, robot commands cannot rely on the cloud). In these three dimensions, the low power consumption and high energy efficiency ratio of edge chips become key variables, and the computing-storage integration architecture can improve the power-energy efficiency ratio by one to two orders of magnitude.
For the embodied intelligence industry, the data bottleneck has been somewhat alleviated. Industry giants have collected massive amounts of video and operational data; algorithms are basically usable, as demonstrated by the showcases of multiple robotics companies at WAIC. The real challenge lies in how to run algorithms efficiently on computing power: model pruning, low power consumption, and low latency must be collaboratively optimized, which is essentially still a system engineering problem.
Hundreds of embodied intelligence startups are testing the waters in different scenarios—logistics, assembly, companionship—but no universally recognized killer application has emerged yet. The underlying chip industry believes that by opening a door to the intelligence of all things for customers, many applications will naturally emerge.
04 Software Ecosystem: A Thicker Barrier Than Chips
If cost efficiency is the endpoint of competition, the software ecosystem is the key roadblock on the way to the endpoint. Returning to the huge gap between theoretical computing power and actual output mentioned at the beginning, its root lies in links such as model selection, fine-tuning, knowledge base construction, middle and lower-level software stack optimization, and server room configuration. Delivering an AI server and implementing an AI project are like buying a wok and cooking a dish; there is still a considerable amount of workload in between.
The core of the ecological barrier for chips is not only a technical issue but also an issue of developer habits. The core challenge faced by new architectures such as computing-storage integration is: not that the chips cannot be made, but that the software ecosystem cannot keep up. Even if the chip parameters are already very impressive, system engineering—adaptation, tuning, and polishing customer scenarios—still requires a massive amount of work.
In edge AI scenarios, the model iteration speed is extremely fast. Cloud-side large models have accelerated from one version every half a year to one version every two months, and edge-side models are also evolving rapidly. Every model iteration requires re-adaptation with the chip, and the speed of engineering implementation directly determines the speed of commercialization. "For a technology to go from being usable to being highly usable, a lot of hard and tedious work is required."
Some viewpoints point out that the middle and lower-level software stack can increase the computing power output of the same hardware by 2 to 5 times, but there were not many in-depth discussions on this direction at this WAIC. Full-stack implementation requires the collaboration of multi-disciplinary professionals: agents, model optimization, knowledge bases, underlying computing power scheduling, and operations and maintenance. Only by assembling these roles is it possible to implement a truly efficient full stack.
When asked whether domestic AI chips most need to develop performance or ecosystem, the answer from an inference chip company was: neither constitutes the current bottleneck; what is most lacking is industrial synergy.
From computing-storage integration to edge NPUs, from VPUs to full-stack integration, companies with different technical routes all point to the software ecosystem rather than hardware performance. There is already an industry consensus on the specialization direction of chip architecture, but companies have substantive differences regarding the speed, depth, and final form of specialization. The underlying logic of the divergence is: is the degree of fragmentation in AI application scenarios sufficient to support the coexistence of multiple specialized architectures? Or will it ultimately converge to a few general-purpose platforms?
When the hardware parameter race approaches the inflection point of diminishing marginal returns, the main battlefield of competition will inevitably shift upwards to the software layer. And this upward shift process is precisely the critical window period for domestic chips to shift from "catching up in performance" to "catching up in the system." Industry insiders further pointed out that the penetration rate of the domestic ecosystem is already very high; universities teach directly on the domestic ecosystem, and customers in key fields choose the domestic ecosystem. This top-down ecosystem construction path is a unique advantage of China.
A further judgment is: domestic AI chips do not constitute the current bottleneck in either performance or ecosystem. This judgment would have been almost unimaginable three years ago. However, while chip parameters are already very impressive, system engineering still requires a massive amount of work. The gap is not in "whether it can be done," but in "how well it is done." A model that scores 70 or 80 points can already do the job, but doing the job well is not a problem for a single model or a single chip; it requires comprehensive follow-up from toolchains, compilers, scheduling systems, and industry adaptation. From AI inference chips to true implementation, a large amount of industrial collaboration is needed: solution cooperation, user scenario cooperation, and supernode network cooperation. Every link requires someone to do the work.
Samples of full-stack integration do exist. From CPUs to software integration companies and then to hardware manufacturing, the group-based system has advantages in large-scale full-stack engineering in key fields. However, this capability is not universal in the industry, and more companies are still fighting alone.
From the customer side, according to industry feedback, it usually takes 6 to 24 months for data center chips to go from initial contact to large-scale deployment, with long testing cycles and high introduction thresholds. This means that even if chip technology meets the standards, commercial implementation still requires a long validation cycle. This cycle cannot be compressed by single-point technology; it requires deep collaboration among chip manufacturers, software suppliers, and customers.
In 2026, the AI chip industry is experiencing a subtle shift: the main battlefield of competition is migrating from wafer fabs and tape-out workshops to the compiler, toolchain, and system scheduling layers. When multiple companies, starting from different technical routes, unanimously point to the same gap, this is no longer the judgment of a single individual, but a collective industry cognition.