Author: Liuqian
A machine sitting on a desk, not much larger than a set-top box, packed with hundreds of GBs of unified memory, capable of running hundred-billion-parameter large models even without an internet connection. No server room, no racks, plug and play, and data never leaves the room. This is the AI Box, a data center moved right onto the desk.
In the AI Box race, Apple, NVIDIA, AMD, Microsoft, and Intel have all entered the fray. But this form factor is not new. Why has the little box that "becomes a computer once connected to a monitor" become a fiercely contested battleground in 2026?
There are two ways to use large models: call the cloud, or run them locally.
The cloud means unlimited AI capabilities, no need to manage the models yourself, and the most powerful capabilities are available anytime. But the costs are also very clear: in subscription models, ChatGPT Pro and Claude Max are both capped at $200 per month; beyond the cap, it is pay-as-you-go. The newly released GPT-6 Astra on September 3 costs $50 per million output tokens. For heavy users running agents, a single long task can output tens of thousands of tokens, making it common to burn through hundreds to over a thousand dollars a month.
Industry insiders note that over the past two years, most enterprises adopting AI have been stuck in the exploratory phase "from concept to implementation": testing scenarios, changing processes, and tuning prompts. Every minute in this phase costs money on tokens, while bosses often only provide direction without specific requirements. "A lot of money is burned, but what are the results? Nobody knows."
The logic for local deployment therefore holds: most mainstream models are open-source and free (DeepSeek and Qwen series are both open-sourced under permissive licenses), hardware is purchased once, there are no subsequent token bills, and data never leaves the server room. For pre-research, validation, and repeated trial and error, the costs are fixed.
Thus, the AI Box was born.
01. What Kind of Box Can Be Called an AI Box?
Local AI deployment has a hard threshold: the model must first "fit in," and memory is the ticket to enter the AI Box arena.
The prerequisite for running large models is loading all weights into memory or VRAM. The full-blooded version of DeepSeek-R1 has 671 billion parameters; full precision (FP16) requires about 1.3TB of VRAM, FP8 quantization requires about 0.7-0.8TB, and even 4-bit quantization requires 400-480GB8. For comparison, a single RTX 5090 has 32GB of VRAM9, and a high-end laptop has 24-32GB of unified memory—enough only to fit small models in the 7-billion-parameter range.
This is the biggest technical proposition for desktop AI devices: packing as much unified memory as possible into a machine that fits under a desk, feeding the model with the highest possible bandwidth, and keeping the power consumption within the limits of an office wall outlet.
Capacity determines how large a model can be run, bandwidth determines how many people can use it simultaneously and how fast the text is generated, and power consumption determines whether it can be placed in an office rather than a server room. Across these three dimensions, the most dramatic thing is that the company best prepared for this was not originally designed for AI.
In 2020, Apple transitioned the Mac to its self-developed M-series chips, making a decision that seemed quite radical at the time: the CPU, GPU, and Neural Engine share the same memory, where memory is VRAM. This architecture, called Unified Memory Architecture (UMA), was originally intended to eliminate the overhead of moving data between chips and reduce power consumption.
Five years later, as the era of large models arrived, this architecture happened to hit all the demand points perfectly.
Today's Mac Studio, in the M3 Ultra version, can be configured with up to 512GB of unified memory and a memory bandwidth of 819GB/s, with a maximum sustained power consumption of 480W10; the M5 Ultra version released on September 22 pushes the bandwidth further to 1.2TB/s, starting at CNY 46,999 for the China mainland version. According to Apple's official statements, this machine can run models with over 600 billion parameters. Reportedly, over the past few months, OpenAI has purchased tens of thousands of Mac minis and Mac Studios for reinforcement learning and agent development, with demand so strong that they have repeatedly urged Apple to expedite shipments. Tim Cook admitted during the July earnings call that "more and more customers are using the Mac mini as a powerful platform for running agents and deploying Mac Studio clusters to run frontier models locally," and noted that Macs have been facing supply constraints throughout the quarter.
02. NVIDIA: Not Selling Machines, Selling the "Same Architecture"
If Apple's entry was "unintentional," NVIDIA's move is a "disruptive strike from a higher dimension" after recognizing the market demand.
The DGX Spark uses the GB10 Grace Blackwell Superchip, featuring 128GB of unified memory, 273GB/s bandwidth, and 1 PFLOPS of compute at FP4 precision, with a total system power consumption of 240W. Officially, a single unit can infer 200-billion-parameter models, and two interconnected units can run 405 billion. The US official website price is $4,699, the initial China mainland price was about CNY 30,000, and after memory price increases, it has approached CNY 40,000.
Looking solely at paper specifications, it has no advantage over the Mac Studio. However, the DGX Spark runs DGX OS (Ubuntu); its software stack is derived from the same source as the 10,000-card clusters in data centers. With the CUDA ecosystem and enterprise-level support, what developers tune on their desks can be seamlessly moved to the cloud.
Around NVIDIA's DGX Spark, seven OEMs—Acer, ASUS, Dell, Gigabyte, HP, Lenovo, and MSI—simultaneously launched whole machines with the same chip. MSI's version is as low as $2,999, and Lenovo's ThinkStation PGX entered the Chinese government and enterprise procurement catalog in July last year. Compared to Apple, NVIDIA has given the AI Box a clearer market positioning and sales ambition.
If the Spark is an entry-level machine for developers, the DGX Station is a desk-side workstation for small teams: featuring the GB300 chip, 748GB of coherent memory (252GB HBM3e plus 496GB LPDDR5X), 20 PFLOPS at FP4, and a total system power of 1600W. Officially, it is claimed to run trillion-parameter models. Although the power consumption has touched the upper limit of ordinary office circuits, it is still an order of magnitude lower than server room servers.
At the Build conference in June this year, Microsoft released the Surface RTX Spark Dev Box: featuring the same GB10 chip as the DGX Spark, 128GB of unified memory, and 1 PFLOPS, with the thermal design power compressed to 100W. Pre-installed with Windows 11 Pro Developer Edition, it is rated to run 120-billion-parameter models locally with a million-token context. It will go on sale in the US in the second half of the year, with media estimates pricing it between $3,000 and $3,500. Its significance in entering the market outweighs its product significance: the Windows ecosystem needs an official answer to tell developers that they don't necessarily have to buy a Mac for local AI.
03. The x86 Camp: Showcasing Cost-Effectiveness in the Inference Era
AMD's Ryzen AI Max+ 395 (codenamed Strix Halo) integrates 128GB of unified memory and 256GB/s bandwidth within a 55-120W power envelope, featuring a 50 TOPS NPU and a total platform compute of about 126 TOPS. In the third quarter of this year, the next-generation Ryzen AI Max 400 (Gorgon Halo) will push the unified memory limit to 192GB, which the company claims can run 300-billion-parameter models.
The trump card of x86 is compatibility. Windows and Linux can be installed directly, and all existing enterprise O&M systems can be reused. The performance may not be the strongest, but not having to change the ecosystem carries the greatest weight in procurement decisions.
Intel has launched Panther Lake (Core Ultra 300 series) with 50 TOPS of NPU compute, packing it into every laptop and mini PC; the NUC 16 Pro, which is closest to the box form factor, has a memory limit of 96GB and is positioned as a Copilot+ mini PC. Notably, Intel's AI Box is not just a box placed on a desk; Intel has also put the AI Box into smart vehicles, giving vehicles that previously lacked sufficient compute a compute boost. It can be seen that Intel has positioned the target of the AI Box in more specific B2B scenarios.
04. AI Box: Who Is Actually Buying Them?
After the boxes are sold, what have they actually achieved?
In the vocational education scenario, schools have turned lesson preparation, grading, practical training Q&A, and lesson plan accumulation into three sets of agents for teachers, students, and management, installed in the boxes for localized operation. The data after the pilot shows: the average weekly lesson preparation time for teachers dropped from 9.2 hours to 5.1 hours, grading time per class was compressed from 2 hours to 0.4 hours, the practical training error rate fell from 22% to 14.3%, and the reuse rate of excellent lesson plans increased from less than 10% to 67%.
Another is the law firm scenario, where case files cannot leave the firm, which is exactly the home turf for the box: similar case retrieval is compressed to within 40 seconds, the automatic generation coverage rate for legal documents is over 80%, and the accuracy rate for conflict of interest reviews is over 95%.
The logic for selecting these scenarios is very clear: repetitive data processing + data security sensitivity + results can be manually reviewed. The advantages of the AI Box are also very obvious. Traditional local server solutions require server room renovation with a cycle of three to six months; public cloud APIs can be activated instantly but carry the risk of data leaving the domain. In contrast, the desktop box is plug-and-play and physically isolated, with deployment time measured in hours.
The shift from selling compute to selling efficiency itself indicates that the market has moved past the stage of "paying for parameters" and has begun to pay for results.
05. Conclusion
The potential of the AI Box is ultimately a matter of economic calculation.
Building a self-owned server room requires a whole machine with 8 RTX 4090s, quoted at around CNY 350,000 on wholesale platforms. The peak power consumption of the whole machine is over 4800W, requiring a server room, cooling, and engineers for installation and maintenance; it absolutely requires an independent server room.
Cloud AI capabilities are unlimited, but the token bill grows linearly with usage, making the costs during the exploratory phase uncapped.
The AI Box ranges from thousands to tens of thousands of CNY, with a power consumption of 100-480W. It is plug-and-play, models are free, and the cost of trial and error is fixed. Compared to an 8-card server, the power consumption differs by 10 to 30 times.
The problem facing the industry is that the ROI is still unclear. Procurement parties who actually pay the money need to recalculate it based on their own processes. Scenarios like customer service, which can be converted into "how many fewer seats," are easy to quantify, but so-called process optimization and efficiency improvements cannot be well measured. Furthermore, similar to the liability issues in the advancement of autonomous driving, if the AI makes a mistake in actual work, who will check it and who will be responsible remains unanswered.
The reality is that there are currently only two mature scenarios: coding and office work, with most other scenarios still being explored. Hardware is just the ticket: data governance, fine-tuning, and iteration operate on a three- to six-month cycle. Once the model changes, the upper-layer applications will most likely need to be reworked. Another point to consider is that, generally speaking, the box cannot be upgraded; expanding capacity means buying another machine. In the context of high DRAM costs, selling boxes seems to have become a way to digest expensive storage. For the market, what they need to see more is what the AI Box can actually do.
The AI Box is not here to replace the cloud. For tasks requiring the most powerful models, elastic compute, and zero maintenance, the cloud remains the optimal solution. The box caters to another type of demand: pre-research with repeated trial and error, data that is unwilling to go to the cloud, fixed compute for small teams of around ten people, and small businesses that can clearly calculate electricity and depreciation costs.
The emergence of the AI Box is about miniaturizing the hardware to achieve compute freedom after paying a fixed cost. Ultimately, the future of the product lies in how compute can be deployed and how it can deliver real value.