In current chip design, cloud-based large models primarily rely on ultra-large-scale clusters of 10,000 to 100,000 cards to provide nearly infinite computing power. However, on edge devices closely related to daily interactions, AI (Artificial Intelligence) deployment must operate within strict power and thermal boundaries, battery energy constraints, and the heavy limitations of the "memory wall and bandwidth wall."
At WAIC 2026, Zhu Xiaoming, CPU Product Director at Arm China, shared the topic "STAR CPU, Helium, and Ethos Empowering AI for Everything," which essentially explores a typical chip engineering problem: maximizing the AI computing potential of embedded devices under limited power and computing resources.
Figure | Zhu Xiaoming, CPU Product Director at Arm China; Source: Photographed by EEFOCUS
AIoT Computing Architecture Transformation: Four Driving Forces
Transitioning from traditional IoT (Internet of Things) to AIoT, the core of the system is shifting from "connecting everything" to "cognizing everything," accompanied by a corresponding transformation in computing architecture. In the past, chips were only responsible for data collection, with all computation transmitted back to the cloud; now, the edge and endpoint must handle a certain degree of autonomous computing and decision-making.
Zhu Xiaoming pointed out that there are four clear engineering reasons behind this transformation:
First, the focus of sensory data is shifting to the edge and endpoint. If the high-frequency, massive data directly generated by the physical world were all transmitted back, it would be unsustainable in terms of economic costs and network bandwidth.
Second, the requirements for privacy and data sovereignty dictate that sensitive data must be processed locally at the source of generation.
Third, the need for millisecond-level real-time interaction. Cloud models can tolerate a few seconds of reasoning delay in the chain of thought, but closed-loop decision-making in edge control systems requires millisecond-level responses; any network latency will directly ruin the user experience.
Fourth, the maturation of low-power, lightweight chip technology. Milliwatt-level NPUs can now deliver TOPS-level computing power within the power budget of an MCU (Microcontroller Unit).
Furthermore, algorithm evolution is simultaneously accelerating this trend. Data shows that two years ago, reaching the 60% minimum commercial threshold in the MMLU test required model sizes of hundreds of billions of parameters. By 2026, thanks to algorithm optimization, small models with around 1B parameters (such as Alibaba's Qwen3 1.7B) can cross this commercial threshold. The "slimming" of large models has made it a reality for edge chips to run them.
Technical Evolution Path of Heterogeneous Architecture
To address the trade-off between edge market demands and constrained resources, architecture design typically follows three evolution paths: CPU instruction extension (efficient combination of general-purpose and inference computing), general-purpose low-computing-power NPU (easy to generalize, ecosystem-friendly), and SRAM in-memory computing (providing higher energy efficiency for specific algorithms).
Zhu Xiaoming revealed that in its hardware layout, Arm China integrates "general-purpose computing" with "inference computing" by taking the CPU instruction extension path. The reason is that this design ensures the general-purpose CPU can maintain indispensable control and logic capabilities while also accommodating AI inference acceleration.
The key to this path lies in Arm Helium Vector Extension technology (MVE, M-Profile Vector Extension). Based on the v8.1-M 32-bit general-purpose architecture, it provides fixed-length 128-bit vector processing capabilities and adds nearly 150 dedicated instructions for edge AI, including 8-bit vector dot product and complex math support.
Based on the empirical data demonstrated by Zhu Xiaoming, the performance improvement brought by Arm Helium technology is highly intuitive:
In the audio codec (LC3) scenario, using the classic Cortex-M4 as the baseline, the most lightweight Cortex-M52 shows a 30% performance improvement, while the high-end Cortex-M85 achieves an improvement of up to 65%.
In machine learning (ML) model inference, compared to the Cortex-M7 widely used in industrial MCUs, the M55 and M85 supporting Arm Helium demonstrate an average 3x performance increase, and in specific cases, even over 8x, in typical TinyML test cases such as keyword spotting and visual wake words.
Figure | Performance test of Arm Helium technology in machine learning scenarios; Source: Photographed by EEFOCUS
Self-Developed "STAR" CPU Roadmap and "STAR 300" Prototype Platform Validation
As the localizer inheriting this architectural DNA, Arm China's self-developed "STAR" series CPU IP is evolving along this timeline.
In 2019, the company released the first-generation STAR-MC1 based on the v8.0-M architecture. Starting from the second-generation STAR-MC2 (corresponding to Cortex-M52), its M-core CPU has fully inherited support for Arm Helium technology. The subsequently released STAR-MC3 further optimized for the AHB bus.
Zhu Xiaoming emphasized that to address the excessive occupation of the CPU by complex, heavy matrix computing or large model execution, the company's R&D team adopted a heterogeneous collaborative design approach. They integrated the self-developed STAR-MC2 CPU with the Arm Ethos-U55 NPU, deployed it on an FPGA prototype validation platform for system-level verification, and launched the "STAR 300" AIoT prototype platform.
Figure | Arm China "STAR 300" AIoT prototype platform; Source: Photographed by EEFOCUS
In this heterogeneous system, the Ethos NPU primarily takes on the role of "computing power supplementation," responsible for offloading specific tasks from the CPU, providing high energy-efficient computing support of up to 4 TOPS.
In current platform empirical tests, this heterogeneous platform has successfully run through six major typical edge AI use cases, including face detection (real-time and pre-recorded), speech recognition (Wav2letter, Conformer), keyword spotting, and image super-resolution.
In actual demonstration interactions, the advantages of the heterogeneous offloading mechanism are quite obvious: in a smart access control system, if edge computing relies solely on a single CPU, the system can usually barely detect a single face; however, when the "STAR 300" platform is connected to the Ethos NPU, the computing power offloading effect is significant. Even when facing 5-6 faces on-site, after the system captures data via a camera and transmits it via an Ethernet cable, it can still accurately and in real-time detect and mark all face information, ultimately presenting the real-time video feed on an LCD screen.
Conclusion
In the design and development of edge AI, the iteration of chip architecture is essentially a constrained balance among power consumption, chip area, and bus bandwidth.
In this regard, Zhu Xiaoming pointed out that through the instruction set expansion of the self-developed STAR CPU, coupled with the hardware dynamic offloading of external dedicated NPUs, this heterogeneous computing system enables lightweight TinyML operators to be executed locally within the processor, while handing over heavy matrix operations to dedicated acceleration units. This dynamic allocation based on actual workload characteristics can provide a low-cost and real-time hardware architecture reference for the engineering implementation of next-generation embedded access control, edge sensing chips, and various micro edge models.