In promotional materials for autonomous driving chips, TOPS has almost become a standard specification. The numerical gap between different chips is extremely wide, ranging from dozens to hundreds or even over a thousand TOPS. However, when applied to actual mass-produced vehicles, a chip with higher computing power does not necessarily mean a more capable system.
This is because TOPS only measures how many operations a chip can complete under specific conditions. What autonomous driving requires is a complete hardware and software system that can ultimately output how many useful computational results under real-world data, real models, and real latency constraints. Therefore, by 2026, simply comparing TOPS is becoming increasingly difficult for judging the true capability of an autonomous driving chip.
What truly deserves attention is actually how much of the nominal computing power can be converted into effective computing power.
01. Why TOPS Cannot Directly Represent Performance
TOPS actually represents an indicator of a chip's theoretical integer operation capability. For example, if a chip is rated at 1000 TOPS, it means that theoretically, it can complete approximately 1,000 trillion operations per second under specified precision and calculation methods. But this number has a very important premise: theoretical conditions.
Different manufacturers do not have a completely consistent statistical caliber for TOPS. Some base it on INT8 computation, while others provide data for INT4, INT8, or even FP16 simultaneously. Some figures represent the peak sum of AI acceleration units across the entire SoC (System on Chip), while others merge the capabilities of different computing engines. Therefore, when seeing 1000 TOPS and 500 TOPS, one cannot simply assume that the former definitely offers twice the performance of the latter.
Besides the statistical caliber, there is another easily ignored benchmarking variable: Batch Size. In AI inference, Batch Size refers to the number of samples processed in a single batch during one forward pass of the model. When chip manufacturers run impressive TOPS numbers in the lab, they generally set a very large Batch Size (e.g., 128 or 256) to maximize the utilization of AI computing units. However, autonomous driving is a low-latency real-time system. To reduce perception lag, inference almost mandatorily uses Batch Size=1 (single-frame, single-inference). Under Batch=1, a large amount of computing power idles while waiting for data loading, and the measured effective throughput is sometimes only 20% to 40% of the nominal peak.
Comparing TOPS directly without asking about the Batch Size is equivalent to swapping mass-production concepts with lab data. When neural networks actually run, they do not keep AI computing units at 100% utilization all the time. Data movement, memory bandwidth, cache hit rate, operator support, scheduling between CPU and NPU, and data conversion between different modules can all become bottlenecks. In other words, a chip may have a theoretical peak of 1000 TOPS, but the computing power actually continuously used by autonomous driving algorithms might only be a fraction of it.
02. Where Does the Gap Lie Between Nominal and Effective Computing Power?
Effective computing power can be understood as a concept closer to actual engineering results. But before elaborating on this, a common misconception needs to be clarified. While compiling this article, the editor came across a formula: 'Effective Computing Power ≈ Nominal Computing Power × Computing Utilization × Precision and Model Matching Efficiency × Memory and Data Movement Efficiency × Software Stack Efficiency.' However, no such simple multiplication formula exists in the engineering field because these factors are highly coupled and mutually influential. Insufficient memory bandwidth directly leads to idle computing units (decreased utilization), and operator fusion optimization in the software stack essentially reduces the number of data movements rather than simply improving an isolated metric. Therefore, effective computing power can only serve as a qualitative analytical framework and cannot be applied as a mathematical formula.
The gap between nominal and effective computing power is actually reflected in three aspects. The first is utilization. If an NPU can peak at 1000 TOPS, but the average utilization during actual model operation is only 50%, then from the perspective of actual computing throughput, it is hard to simply treat it as a 1000 TOPS chip.
The second is model matching. Different AI accelerators vary in their support for Transformer, CNN, BEV, Occupancy, temporal models, and various custom operators. A chip may have high theoretical computing power, but if a large number of operators in the model cannot be efficiently executed on dedicated AI units, they generally need to fall back to the chip's built-in GPU general-purpose computing units or DSP for processing. In such cases, the peak computing power is hard to fully translate into actual performance.
The third is memory bandwidth. Even if utilization and model matching are perfectly optimized, insufficient memory bandwidth will make the movement of model weights and intermediate feature maps a bottleneck. AI computing units will still have to wait for data, and the peak computing power cannot be realized either. These factors will ultimately be uniformly reflected in end-to-end measured results. This is also why, for the same end-to-end model, the FPS, latency, and power consumption run on different chips can vary significantly.
To measure effective computing power, ultimately, one must look at the chip's end-to-end throughput and latency performance on real models.
03. What Does Autonomous Driving Value More in a Chip?
For autonomous driving, what truly matters is not how many multiply-accumulate operations a chip performs per second, but whether it can stably complete tasks such as perception, prediction, and planning within a specified time window. For example, when running the same end-to-end model, one chip can achieve 30Hz while another can only achieve 15Hz. Even if the TOPS gap between the two is not particularly large, the actual system performance may still show significant differences.
This is because autonomous driving is a real-time system. Camera inputs, LiDAR point clouds, millimeter-wave radar data, and vehicle status information continuously enter the system. The algorithms need to complete data preprocessing, feature extraction, fusion, inference, and output within a limited time.
Therefore, the truly important metric for autonomous driving chips is actually whether computing power, bandwidth, latency, and power consumption can simultaneously meet the requirements in a Batch Size=1 scenario. Especially after entering the end-to-end stage, as model sizes continue to expand, the computational and memory access pressures brought by Transformers and temporal modeling are becoming increasingly obvious. At this point, looking solely at TOPS will be even more unreliable. Even if a chip has higher peak computing power, if the memory bandwidth is insufficient or the on-chip cache design is unreasonable, it may still face a situation where there is abundant computing power but the NPU cannot be fully fed.
04. What Truly Creates the Gap Is Often Beyond the Chip Itself
By 2026, it is becoming increasingly difficult to evaluate autonomous driving chips in isolation from the software stack.
Compilers, operator libraries, model optimization toolchains, quantization schemes, memory scheduling mechanisms, and runtime systems will all directly affect the final performance. The same model may exhibit completely different execution efficiencies on different chips. For example, after a model undergoes structured pruning, operator fusion, and INT8 quantization, the theoretical computational load may decrease. However, if the chip lacks good hardware support for these optimizations, the final latency may not decrease synchronously.
Conversely, if a chip possesses mature compilers and operator optimization capabilities, even if its nominal TOPS is not the highest in the industry, it may achieve better actual performance through higher computing utilization.
Therefore, evaluating an autonomous driving chip now is actually evaluating the overall efficiency of hardware + software + models.
05. Is Effective Computing Power Under Low Power Consumption More Valuable?
There is another easily ignored issue with autonomous driving chips: power consumption. AI chips on desktop servers can trade higher power consumption for performance, but automotive chips must consider overall vehicle energy consumption, heat dissipation, and long-term stable operation. Therefore, simply comparing peak TOPS is becoming increasingly meaningless; what is more worth comparing is effective computing power per watt. For example, one chip has a very high peak computing power but requires massive power to reach that peak, while another chip has a slightly lower peak but can continuously run the same model at a lower power consumption. For mass-produced vehicles, the latter may actually be more valuable.
Especially as advanced intelligent driving systems are developing towards larger models, longer temporal sequences, and more complex multimodal inputs, the demand for computing power continues to grow, yet power consumption cannot increase indefinitely. Therefore, the core of future chip competition is highly likely not about who has the highest TOPS, but who can convert more TOPS into truly useful model inference capabilities at a lower power consumption.
06. How Should Autonomous Driving Chips Be Compared?
By 2026, if directly comparing two autonomous driving chips, one should not just look at TOPS, but should at least consider several dimensions together.
The first is theoretical computing power under unified precision. It must be clarified whether it is INT4, INT8, or FP16 to avoid direct comparison under different statistical calibers. At the same time, one should also ask whether this is a benchmark score under an ultra-large Batch Size or a single-Batch measured result close to mass production.
The second is actual model throughput and latency. This includes real test results for typical perception models, BEV, Occupancy, end-to-end models, and temporal models, and must include the time consumed by pre/post-processing, because in many cases, this proportion even exceeds the inference itself. Furthermore, one must also look at memory bandwidth, on-chip cache, AI computing unit utilization, and data transmission efficiency among CPU, GPU, and NPU.
The last is the software ecosystem, which includes compilers, operator coverage, quantization toolchains, model migration costs, as well as stability and power consumption performance after mass production. Only through such comparisons can the result be closer to the actual computing power the chip can truly provide to the autonomous driving system.
07. Final Words
In the future, the competition for autonomous driving chips will highly likely gradually shift from stacking how many TOPS to how many TOPS can be truly utilized. Nominal computing power solves how much a chip can theoretically compute, while effective computing power solves how much work the actual system can ultimately complete. The gap between the two precisely determines the true engineering value of an autonomous driving chip.
Therefore, the next time you see a chip claiming to have hundreds or even over a thousand TOPS, the real question to ask is not whether this number is high enough. Instead, you should ask: What model is being run? What is the actual utilization rate? What latency is achieved at what power consumption? These three questions are often closer to the true capability of an autonomous driving chip than the TOPS itself.