Author | Qin Qianwen, Editor | Liu Wei
On October 20, DeepSeek open-sourced DeepInfer, the framework that supports its large-scale inference services for V4.1-Flash.
In the past, the common practice was for model providers to open-source their weights, allowing users to run them using third-party frameworks like VLLM and SGLang, while the inference systems used for their own online services were rarely shared externally. However, what DeepSeek has open-sourced this time includes MoE expert parallelism, dynamic load balancing, speculative decoding, and asynchronous scheduling. These are all inference technologies that DeepSeek has refined over a long period in its own production environment to make large models run faster and more compute-efficient.
This move has excited many users in the open-source community, but within the industry, it has also sparked massive discussion and controversy.
01. Inflicting a Wound on Oneself to Let Others Make Money?
In the past, model providers translated internal cost reductions into their own pricing advantages. Now, with this move, DeepSeek is directly spreading its inference efficiency to third-party deployments.
This is a direct benefit to the model supply market. Industry insider Gu Yuan pointed out that DeepSeek presents a dilemma for cloud providers: although the model weights are open and third parties can deploy them independently, lacking the original provider's inference engineering accumulation, it is difficult for them to compress service costs to the extremely low prices of the official API, which leads customers to purchase directly from DeepSeek.
Unable to offer a price advantage, cloud providers need to rely on capacity, service guarantees, or other supporting capabilities to win orders. Simply reselling the same model makes it hard to generate profit margins, which is why some cloud providers are privately reluctant to sell DeepSeek's models. The core issue here is that even after the model weights are open-sourced, the engineering capability to run the model at low cost remains in the hands of the original provider.
Another industry practitioner, Max, commented, "Only those who have truly wrestled with Infra can genuinely understand the gold content of this open-source release. Although what is open-sourced this time is the offline inference version, and the KV Cache management and load balancing components required for full online services have not all been made public, and the currently supported hardware is also limited, this is already quite remarkable."
Running a model with hundreds of billions of parameters stably, efficiently, and at low cost actually requires years of engineering accumulation. From operator optimization and VRAM management, to expert parallelism and communication overhead, and then to load balancing and inference scheduling, these links need to be repeatedly refined, and various bizarre compatibility issues between hardware and software must be resolved. Many times, even with the same GPUs running the same model, if the efficiency of the inference framework falls short, the cost difference can be substantial, and these gaps ultimately translate into real money.
It is important to note that the most expensive expenses for large model companies are almost entirely concentrated on these GPUs. Previously, an article by Leiphone mentioned that the actual utilization rate of GPUs in the industry has long been low, averaging only 25% to 27%, meaning that more than half of the purchased compute power is not truly utilized. AMD once disclosed that under a similar eight-GPU MI355X hardware configuration, through software optimizations such as operators, parallelism, and scheduling, the average inference throughput of the three tested models reached approximately 3.3 times that of the old software solutions. This is also a crucial reason why model providers work hard to tackle Infra engineering.
Currently, the focus of competition in the entire large model industry has shifted from basic pre-training to engineering implementation and inference optimization, requiring each GPU to generate greater value. Now, DeepSeek has instead put its internal inference framework, DeepInfer, which supports its own services, on GitHub, further opening up a batch of engineering capabilities for achieving low-cost inference, and explicitly stating that this is the key to DeepSeek achieving model pricing efficiency. Essentially, DeepSeek has taken out its money-making craft and allowed others to compete with it.
With the help of DeepInfer, third-party service providers can enable the same batch of hardware to process more requests, bringing down the cost of running the same model. This cost saving can be used to improve profits and also provides room for lowering API quotes. For most service providers, as long as they can buy GPUs, they can build the underlying infrastructure and deploy models, and can even achieve API prices lower than DeepSeek's official prices, while still making money from the API price difference.
In addition, boosted by the open-sourcing of DeepSeek's inference framework, the job market capacity for Infra engineers may further expand. Industry insider Fang Sheng believes that in the past, only large companies could afford to hire top-tier Infra talent, with monthly salaries nearing 100,000 CNY, which small and medium-sized companies could not afford. But under a series of mainstream open-source frameworks such as VLLM, SGLang, Token Speed, and DeepSeek V4, small and medium-sized companies can now use the tools that were previously only affordable for large companies.
02. Competition Shifts to Deployment Costs, Price War Intensifies Instead
Over the past few years, domestic open-source models have attracted a large number of developers with their lower API quotes and the flexibility of allowing enterprises to deploy them independently. However, today, overseas closed-source providers are also starting to sell more powerful economy models in similar price ranges.
Just this month, Anthropic slashed the price of Haiku 5.5 for short requests to USD 0.1 per million input tokens and USD 0.5 for output, which are only one-tenth of Haiku 4.5's prices respectively. OpenAI's price cuts were also straightforward. For the GPT-6 Luna released on September 22, the input price dropped from USD 0.2 in the previous generation to USD 0.1, and the output price dropped from USD 1.2 to USD 0.5.
Now, with Chinese open-source providers led by DeepSeek further reducing deployment costs, this situation will spawn more service providers to join the competition, thereby increasing the pricing pressure on other model providers. Leiphone learned that some industry insiders have a very intense attitude towards the open-sourcing of DeepInfer: open-sourcing model weights is completely reasonable, but open-sourcing infrastructure means plunging the industry into a more fierce price war.
"Cheap customers have no loyalty. Opening up all your technology will only result in Luna, Haiku, and other domestic model providers jointly crushing you once again at the cost-performance level." They believe that currently, there is no visible PMF (Product-Market Fit) that open-sourcing can generate. Without SOTA (State-of-the-Art), there is no voice. Unless a finished product at the Opus/Sol level is developed first, it is truly meaningless except for handing bullets to competitors. These concerns are not unfounded.
Previously, the CEO of Broadcom presented revenue data for current language models at a Goldman Sachs conference: in terms of the number of generated tokens, closed-source and open-source models each account for roughly 50%, and the inference costs of the two types of models on servers are also similar. However, the annual revenue of closed-source models is around USD 120 billion, which is four times the USD 30 billion of open-source models.
On X, a user directly called out: Is DeepSeek not interested in making money, but just wants to force AI companies worldwide to reprice? It may not necessarily make money through APIs itself, but the first to get hurt are the frontier model providers that follow the price cuts. They must not only maintain the call volume for routine tasks but also find demands that can support high prices.
In 2026, the product lines of mainstream frontier model providers have formed two divisions of labor: economy products like Flash and Haiku emphasize high concurrency, low latency, and low cost to expand application-layer calls; flagship products continuously upgrade multimodal, long-context, and complex reasoning capabilities to compete for high-value demands.
Under this context, frontier model providers hope to expand user coverage and revenue scale while maintaining leading model performance, but the reality is that the user base willing to pay for high-priced products remains limited.
The consumer AI report released by a16z in October mentioned that users of Claude's highest-priced personal plan, Max, account for 7.3% of consumer paying users; for the hundred-dollar-level high-priced plans of Google and ChatGPT, the proportions are only 1.3% and 1.1% respectively. In the past, the narrative of AI model companies was mostly built on a premise: the stronger the model, the higher the price it can charge.
But the fact is that China's low-priced, open-weight models are systematically impacting this business model. Industry insider Er Dun analyzed that closed-source model providers are generally in an awkward position.
On the one hand, many application scenarios that can actually generate revenue do not particularly need expensive top-tier models; a combination of mid-to-low-tier models is basically sufficient. However, this type of model is facing a price massacre from open-source models.
On the other hand, the R&D of top models has slowed down its iteration speed due to the exhaustion of existing data, making it impossible to attract substantial investment by improving performance, and unable to achieve a growth closed loop through the logic of continuing to invest in performance enhancement.
Under such circumstances, the crowded model providers will sooner or later usher in a wave of mergers and clearances.
There is no clear answer as to how the large model industry will ultimately be reshuffled, but current Chinese open-source models are still accelerating their iterations: DeepSeek V4.1 Flash continues to push the limits in coding and Agent capabilities, with prices still at rock bottom.
The next round of MiniMax, Qwen, Z.ai, and Moonshot is also poised to launch. These models emphasize being "the strongest at the same scale" more and more, and prices are still being slashed. This round of competition will once again refresh the industry's "kill line."