Since the explosion of Large Language Models (LLMs), AI inference has been almost entirely anchored in the cloud. This centralized architecture is powerful, but network latency is unacceptable for real-time interactions in the physical world, and the compliance risks of data leaving the domain constitute a veto in highly sensitive industries. Therefore, shifting inference capabilities down to edge devices has become an inevitable choice at the industry level.
As the increase in model knowledge density brings 3B-7B lightweight models closer to the performance of early large models, mobile NPU (Neural Processing Unit) computing power has broken through 100 TOPS (Tera Operations Per Second), and 4-bit quantization along with model lightweighting technologies have compressed the storage of 3B models to 1.27GB. The technological tipping point has arrived, and edge AI has entered the stage of large-scale commercialization.
However, constrained by the computing power ceiling, memory bandwidth, and power/thermal design constraints of devices, edge AI still has an order-of-magnitude gap with the cloud in scenarios such as complex reasoning, multimodal reprocessing, and long-context processing. The real question is not "whether edge can replace the cloud," but rather: which tasks are suitable for shifting to the edge, and which must remain in the cloud? Where exactly is the capability boundary of edge AI?
Edge AI vs. Cloud AI: Architectural Differences and Capability Boundaries
The fundamental difference between edge AI and cloud AI lies in the execution location of inference computing. This distinction, seemingly simple, systematically determines the divergence of the two in key dimensions such as latency, privacy, cost, and model capabilities.
Among these, the advantages of edge AI in latency, privacy, and cost are scenario-determinative. Millisecond-level response transforms real-time scenarios such as translation and gesture recognition from "usable" to "feasible"; keeping data on the device naturally meets the compliance requirements of highly sensitive industries like finance and healthcare; and the marginal inference cost approaching zero continuously amplifies the economics of high-frequency invocation scenarios over time.
However, the disadvantages of the edge in model capabilities are equally structural. The 3B-7B parameter scale has an order-of-magnitude gap compared to the trillion-parameter models in the cloud. Although model miniaturization has gradually narrowed the gap by 2%-3% in tasks like text summarization and intent recognition, there remains a significant gap in scenarios such as complex reasoning, long-context understanding, and high-quality generation.
This also determines that the edge and the cloud are not an either-or choice, but rather edge-cloud collaboration. Latency-sensitive, privacy-sensitive, and high-frequency invocation tasks are shifted to the edge, while complex reasoning and recomputation tasks are routed to the cloud, dynamically scheduled by the system layer. Apple Intelligence, Google Gemini Nano+Pro, and Huawei's Pangu + edge combination all follow this route.
Current Status of Edge AI Implementation
From smartphones to PCs, and from wearables to smart cars, edge AI capabilities are accelerating their penetration across all categories of terminal devices.
AI Smartphones: The Most Likely System-Level Agents to Be Deployed
Smartphones are the first and most core carrier for the explosion of edge AI. The layout of major manufacturers has shifted from "superimposing AI features" to "deep system-level integration":
- Apple: Apple Intelligence achieves cross-device agent operations through edge-cloud collaboration, processing sensitive data on the edge to ensure privacy while providing personalized services.
- vivo: Launched a 3B edge multimodal reasoning large model with a memory footprint of only 2GB. It has been deployed in scenarios such as AI writing for input methods, call summarization, document summarization, and intelligent naming of recordings.
- OPPO: The Find X9 series emphasizes breakthroughs in the edge AI experience, optimizing edge processing for scenarios like photo object removal and intent recognition.
- Honor: The Magic8 series claims to be the first self-evolving AI-native smartphone. In partnership with Qualcomm, it debuted edge low-bit quantization technology, improving inference speed by 15% and reducing power consumption by 20%.
- Huawei: Relying on the deep integration of the HarmonyOS and the Pangu large model, it builds an edge Agent framework.
A notable trend is that manufacturers are no longer blindly stacking parameters. Last year's 7B models required 3.5GB of memory, whereas this year's 3B models have significantly narrowed the gap with cloud models in certain tasks. Advances in model miniaturization have made edge AI truly usable on mid-to-high-end models.
As major manufacturers continuously attempt to implement AI on smartphones, it is believed that edge large models will evolve from "novelty" to "daily use," transforming from single functional plugins into system-level agents that understand user intent, comprehend usage habits, and provide close assistance in work and life.
AI PC: Mostly a Gimmick?
As the core carrier of productivity, ordinary users rarely proactively invoke local large models to process text in daily office work on PCs. Edge AI on smartphones mainly carries system-level seamless services (such as notification summarization and passerby removal), while office scenarios on PCs have extremely high requirements for generation quality and logical coherence, and are not sensitive to millisecond-level latency. This makes "connecting to the internet to invoke cloud flagship models" a more pragmatic mainstream choice.
Despite the massive promotional campaigns by manufacturers, and although companies like Lenovo and Huawei have indeed optimized the experience in document polishing, offline speech translation, and cross-application collaboration by building in local large models, the practical bottlenecks of edge implementation remain significant. Although the computing power of flagship chips such as Qualcomm Snapdragon X Elite and Intel Core Ultra 9 has surged to 45 TOPS and even 99 TOPS, with paper parameters sufficient to support the operation of 10-billion-parameter models, in actual office workflows, constrained by memory bandwidth, thermal and power dissipation, and capability loss after model quantization, edge models still struggle to match the full-blooded large models in the cloud for complex long-text processing. AI on PCs plays more of a role in "privacy fallback" and "offline emergency" rather than replacing the cloud as the productivity hub.
Wearable Devices: Merely Sensory Extensions
Wearable devices such as AI glasses and AI earphones are becoming important carriers for edge AI implementation. In October 2025, the monthly active user scale of smart wearable apps reached 159 million, a year-on-year increase of 12.8%. Huawei, Rokid, Baidu, and Rayneo Innovation have successively launched new AI glasses. IDC predicts that China's smart glasses shipments will reach 4.508 million units in 2026, a year-on-year increase of 77.7%.
Unlike smartphones and PCs, the value of edge AI in wearable devices does not lie in running large models, but in becoming a sensory extension of AI. The cameras and microphones on glasses are extensions of vision and hearing, capable of continuously perceiving environmental information; earphones naturally possess the entry advantage for voice interaction. In real-time translation scenarios, the glasses complete voice wake-up and noise reduction preprocessing on the edge, the cloud or smartphone completes the translation inference, and the glasses then display the translated text in real time.
The AI on wearable devices is responsible for the lightweight closed loop of "perception-preprocessing-output," rather than full-chain inference. This division of labor enables wearable devices to provide a smooth AI experience even under extremely constrained computing power and memory conditions (typically with only hundreds of megabytes of available memory).
However, this positioning also implies limitations: wearable devices can hardly run large models independently and are highly dependent on collaboration with smartphones or the cloud. Once disconnected, AI capabilities shrink significantly. Meanwhile, constrained by battery capacity (typical AI glasses batteries are only 200–300 mAh), battery life is often less than 2 hours when continuously running AI functions, making it difficult to support all-day use.
Therefore, the more accurate positioning of edge AI on current wearable devices is a lightweight perception layer. Data collection and preliminary processing are completed on the edge, while core reasoning still relies on smartphone computing power or the cloud. There is still a considerable distance to go before achieving truly "independent AI terminals."
Smart Cars: Future All-Scenario Spatial Agents
Among terminal devices, smart cars are undoubtedly the carrier with the richest dimensions for edge AI implementation. Its AI evolution presents a dual-track parallel pattern of autonomous driving and smart cockpit, and the two sectors are gradually moving towards cockpit-driving integration to build all-scenario spatial agents.
In the field of autonomous driving, the divergence of technical routes and scenario adaptation have become core highlights. Tesla's FSD v14 relies on an end-to-end pure vision solution to achieve an urban commuting experience close to that of an experienced driver, with the takeover interval approaching the 10,000-mile level, demonstrating the advanced potential of the pure vision route. Among domestic manufacturers, Huawei ADS adheres to the technical route of LiDAR and vision fusion. Targeting complex domestic traffic scenarios, such as vehicle cut-ins and pedestrians suddenly darting out, it possesses higher redundancy and safety, better fitting local road condition requirements.
In terms of the smart cockpit, computing power upgrades have made the implementation of edge large models possible. Cockpit chips represented by the Qualcomm Snapdragon 8295 have an NPU computing power reaching 30 TOPS, already possessing the capability to run 2B parameter-level multimodal models locally. GAC took the lead in deploying a 2B edge multimodal large model on the 8295 platform. By fusing multiple types of perception data such as in-car cameras and microphones, it achieved outstanding performance with end-to-end response latency below 1.6 seconds and in-car target recognition accuracy exceeding 95%, significantly improving the fluency and accuracy of cockpit interaction.
Notably, for smart cars to truly achieve the all-scenario spatial agent with a fully autonomous closed loop of perception-understanding-decision-execution across the full chain, they still cannot do without the support of edge-cloud collaboration. The cockpit edge mainly undertakes the core responsibilities of real-time perception and fast interaction, while the 200B-level large model in the cloud handles high-intensity computing power demands such as complex intent parsing and multi-step task planning. The two deeply collaborate through a four-layer architecture of "memory-planning-scheduling-execution," jointly promoting the evolution of smart cars towards a higher-level intelligent form.
What Other Bottlenecks Remain for Edge AI Implementation?
Although edge AI has incomparable advantages in dimensions such as latency, privacy, and cost, constrained by the hard constraints of physical computing power, memory bandwidth, and energy efficiency ratio, its capability circle currently still presents distinct hierarchical characteristics.
In scenarios of complex reasoning and multimodal reprocessing, the capability shortcomings of edge models are the most intuitive. Although current 3B-7B scale edge models can handle lightweight tasks such as text summarization and intent recognition, in complex scenarios requiring deep chain-of-thought reasoning, such as mathematical proofs and legal contract logic verification, the performance gap with 100-billion and trillion-level models in the cloud is of an order of magnitude. multimodal reprocessing also faces severe bottlenecks; video understanding requires extracting features frame by frame and performing long-sequence modeling, with a computing load about ten times that of pure text processing.
Furthermore, actual test data shows that when a 7B model runs at full capacity on a mobile device, the peak power consumption can reach over 2 amperes, which is twice the power consumption of heavy-load gaming. Under existing battery capacity and passive cooling architectures, such high-load tasks must still be handed over to the cloud for processing in the short term, with the edge playing more of a "perception entry" role rather than a "computing core."
Cost and memory bottlenecks hinder the popularization process of edge AI. Among terminal devices, the performance configuration of smartphones is already relatively high. However, for mid-to-low-end models (with 8GB and below memory), which account for over 60% of the market, the remaining available memory is extremely limited after deducting system overhead. Loading a quantized model often occupies nearly half of the resources, which is highly likely to trigger the system killing background processes or causing lag.
This means that if edge AI wants to penetrate all price segments, it not only relies on extreme lightweighting at the algorithm level but also requires waiting for the cost reduction of chip manufacturing processes and memory bandwidth upgrades. This is a hard constraint of industrial economics, determining that the popularization speed of edge AI will be significantly slower than cloud iteration.
The more core contradiction lies in the fact that the energy consumption paradox constitutes a high wall for "true intelligence." Although continuous learning on edge devices does not require the model to be resident 24/7, the high-frequency inference, real-time perception, and periodic lightweight fine-tuning it relies on will significantly raise the overall energy consumption baseline of the edge. This is especially true for embodied intelligence represented by humanoid robots with strong interactive capabilities, which has a structural conflict with the "battery life first" design philosophy of mobile devices.