By Wei Rong, Edited by Bao Yonggang
Birds migrate in pursuit of a climate suitable for survival.
The migration of an AI computing task, however, might be sent hundreds or even thousands of kilometers away simply because another data center has more abundant and cheaper electricity.
In the past, the operating location of computing power was mostly determined by data center site selection. Today, changes in power supply are beginning to become a new reason for computing tasks to choose their landing spots.
"Power demand usually cannot be moved, but computing tasks can be migrated," said Fu Zhi, Founder and CEO of Gongji Technology.
However, unlike migratory birds, AI tasks cannot complete this migration on their own. The models, data, and operating environments they rely on also need to be properly arranged in another cluster.
Compute-power and electricity coordination aims to make computing power follow electricity, but this migration is far from being as light as the flapping of a bird's wings.
01Computing Power Doesn't Need to Relocate, but Tasks Have Migration Costs
According to previous public reports, a cross-regional compute-power and electricity coordination field test scheduled over 280,000 AI inference requests in three days, reducing the computing power electricity cost per token by approximately 31%.
Similar explorations have also emerged overseas. Google, NVIDIA, and Emerald AI jointly launched the AI Energy Management Alliance, attempting to enable data centers to flexibly adjust their power loads based on grid conditions.
While the power system begins to try mobilizing computing tasks, another group of enterprises is building the infrastructure to allow tasks to flow across clusters.
Leiphone met Fu Zhi, Founder and CEO, at the Gongji Technology office. This young founder, labeled as a "post-00s entrepreneur," was quite straightforward when discussing the frequently mentioned computing power scheduling in the industry.
"The word 'scheduling' has been overused," Fu Zhi said. In his view, the computing power scheduling mentioned by many companies in the industry mainly revolves around a single cluster and leans more towards the management of existing GPU resources. Gongji, on the other hand, focuses on task flow between different clusters. Although both belong to the category of computing power scheduling, their focus is different.
For the migration of computing tasks, Fu Zhi used a simple analogy: "For example, if I need to use your brain's computing power, I certainly can't borrow your brain. So I give you the math problem, you calculate it, and then give me the answer."
The same applies to computing resources scattered in different places—the machines stay in place, and tasks and results are transmitted over the network.
In the past, this cross-cluster scheduling was mainly to match the supply and demand of computing power. One side has idle resources, and the other has computing needs; tasks can be assigned to other clusters for execution.
Compute-power and electricity coordination adds another use to cross-cluster scheduling: when there are differences in power supply, electricity prices, and grid loads in different regions and at different times, the execution location of tasks can be adjusted on demand.
Since 2023, Gongji Technology has begun exploring distributed computing power. Fu Zhi judged at the time that as AI applications increase, future inference tasks will not all be concentrated in large clusters; some demand may be undertaken by small and medium-sized clusters built by different entities in different regions. To organize these scattered resources, it is necessary to enable computing tasks to be allocated across clusters.
Fu Zhi mentioned that he started thinking about another issue in 2023: wind and solar power in some regions of China face challenges in accommodation, transmission, and storage. If it is inconvenient to send electricity to where computing is needed, could we do the reverse and send computing tasks to where there is electricity?
He summarized this idea as "using network cables instead of power lines."
In Fu Zhi's vision, the future distributed computing power network can be further integrated with distributed wind and solar power. Where new energy supply is abundant, more local computing power is called upon; when power is tight, fewer tasks are arranged to be executed locally. (For more information on distributed computing power, feel free to add the author's WeChat treelog10 to discuss)
As the proportion of electricity used by computing power continues to rise, Fu Zhi believes that with the increasing demand for AI computing power electricity, the connection between computing power scheduling and the energy system will further deepen. However, to make computing power truly follow the flow of electricity, merely being able to send tasks to another cluster is far from enough.
02Tasks Migrate Quickly, but the Effort is Spent Before Migration
To send a math problem to someone else, you only need a piece of paper with the problem written on it. So, how much preparation does the scheduling system need to make in advance for the migration of an AI task?
In Fu Zhi's view, compute-power and electricity coordination can unfold along two paths: one is to migrate tasks between different regions, leveraging the differences in power supply and prices across regions; the other is to adjust the computing load within a single data center to adapt to the power supply rhythm of the grid.
Leiphone posed a hypothetical question to Fu Zhi: If an intelligent computing center receives a grid instruction to reduce its power load by 10% within five minutes, what should it do?
Fu Zhi provided two solutions: if there is coordination with other data centers, some computing tasks can be transferred out; if it can only rely on self-regulation, the throughput speed of tasks can be appropriately reduced to decrease power demand. He prefers the former because the impact on user experience is relatively small.
But to have other data centers take over tasks in a short time, the computing power scattered in different places must first be organized.
Xu Zhongziheng, CTO of Gongji Technology, introduced to Leiphone that the access conditions for different clusters vary greatly. Different clusters may run on bare metal servers, cloud platforms, or the production environments of third-party enterprises, with varying open interfaces, management permissions, and hardware conditions.
The scheduling platform must complete the access based on these differences and try to avoid interfering with the original business. "This actually determines how many GPUs you can ultimately connect to," Xu Zhongziheng said.
Obtaining access permissions is the first step; whether a task can run quickly depends on whether the models and data are already prepared.
"The data accompanying a task may not be freely schedulable; it has strong gravity," Xu Zhongziheng stated. For example, an AI business might require tens of gigabytes of software images. If downloading only starts after the task arrives at another cluster, it will be difficult to start quickly. Gongji uses methods such as image preheating, model distribution, and caching to prepare the required files in the target cluster in advance.
Xu Zhongziheng also mentioned that an AI creation platform Gongji contacted has a model repository of over 10TB, containing tens of thousands of models. To switch businesses between different clusters, model distribution must be completed in advance, enabling the target cluster to access the required files in a timely manner.
Once these preparations are completed, the migration of some online inference tasks can be mainly achieved by switching subsequent request traffic, without having to retransmit large amounts of model files every time. "We have already paid this network bandwidth cost in advance," Xu Zhongziheng said.
Switching between different hardware also relies on prior preparation. Gongji has carried out model adaptation for computing platforms such as NVIDIA, Ascend, and Hygon. Xu Zhongziheng believes that the most heavy work in heterogeneous scheduling occurs during the adaptation phase; once completed, the resource allocation itself is relatively simple.
For Gongji, completing model and hardware adaptation in advance is just the first step. To enable more tasks to flow between different clusters, the scheduling platform also needs to continuously expand the range of connectable computing power resources.
Fu Zhi described today's digital world as the "pastoral era." "Today, our real world is already a concrete jungle, with skyscrapers, very modernized. But the virtual world still belongs to the pastoral era; its highways and its networks haven't been built yet," Fu Zhi said.
In his view, building a distributed computing power network might be a business that lasts for decades, but the opportunity window in the early stage of the industry might only be two or three years.
Whoever can take the lead in connecting enough computing power resources and occupying key positions in the network will have a better chance of gaining an advantage in future competition.
03Electricity Prices Change, but the Scheduling Platform May Not Perceive It
Building a distributed computing power network is a competition to seize the initiative. But even as this network is gradually built, whether computing tasks can truly follow changes in power supply still depends on how much commercial benefit participants can gain from it.
Regarding the commercialization dilemma of compute-power and electricity coordination, Fu Zhi first talked about the economic motivation for data centers to participate in load regulation. He believes that in China, the revenue obtained by many data centers through adjusting computing loads is not yet obvious.
If the investment in related equipment and scheduling exceeds the saved electricity bills, operators will lack the motivation to participate proactively. In Fu Zhi's view, this is also one of the reasons why large-scale commercial closed loops for compute-power and electricity coordination have not yet formed in China.
The differences between the Chinese and US markets have also led compute-power and electricity coordination down different exploration paths.
Xu Zhongziheng also talked to Leiphone about this difference. He introduced that many domestic data centers currently use contracted electricity prices. In his view, the US market is currently more focused on how to cope with power supply constraints, while the domestic market, as the power market mechanism further improves and price signals are transmitted to the computing power operation links, will pay more attention to electricity price forecasting, load forecasting, and how to reduce the average electricity cost of computing power operations.
Technically, the "using computing to replace storage" proposed by Fu Zhi previously, in addition to transferring computing loads across regions, also includes adjusting the execution rhythm of tasks to adapt to changes in power supply in the time dimension. But being able to achieve load regulation technically does not mean that the scheduling platform already has the ability to allocate tasks based on electricity price changes.
Xu Zhongziheng introduced that currently, when allocating tasks, Gongji mainly considers the remaining resources of the cluster and the cost of computing instances. If a cluster has idle computing power and the rental price is relatively low, the platform may prioritize arranging tasks there.
However, the cost here is still the computing power usage price, not the real-time electricity price of the grid. "We have not yet achieved the true realization, which is to make the GPU-hour cost perceive the real-time electricity price of the grid," Xu Zhongziheng said. Even if the electricity price drops in a certain place, if this change is not reflected in the computing power instance quotation, the scheduling platform may not necessarily have the motivation to transfer tasks there.
As mechanisms such as the domestic electricity spot market gradually develop, the differences in electricity prices at different times and in different regions may provide new cost optimization space for computing power scheduling. But to make these differences truly affect the destination of tasks, changes in electricity prices need to be further transmitted to the computing power operation and trading links.
Xu Zhongziheng believes that in the future, computing power operators may need to combine electricity price and load forecasting to decide when to increase computing, when to reduce operating speed, or when to arrange some tasks in regions with lower electricity costs.
At that time, whether computing power scheduling can further consider energy costs will depend on the extent to which electricity price changes can be transmitted to the computing power operation and trading links.
The autumn wind is blowing, and migratory birds have already embarked on their journey. For computing power to truly follow the flow of electricity, it still needs to cross the thresholds of cost and market beyond technology.