In 2026, XPeng's second-generation VLA introduced a 30-second historical context while incorporating a 6-second future prediction. This raises an interesting question: since driving is essentially about deciding what to do next, why does the model need to constantly look back at what has already happened?
01. Why Is Looking Only at the Present Not Enough?
Suppose there is a vehicle ahead, currently 40 meters away from your car. Looking at this single frame, the autonomous driving system can detect the vehicle ahead and estimate its current spatial state, such as its position, but it is difficult to judge exactly how it will move next based solely on this instant. However, the situation changes if information from the past few seconds is incorporated. The vehicle might have been maintaining its lane, then gradually drifting to the side, and subsequently starting to decelerate.
Viewing the current frame in isolation, it is merely a vehicle 40 meters away; but when continuous states are linked together, the system perceives a traffic participant undergoing changes. This is the true value of historical information. It is not to let the model know what happened in the past, but to let the model understand how the current state was formed through changes. There are numerous similar situations in driving scenarios.
A pedestrian reappears in front of the vehicle; the current frame can only tell the model that a pedestrian has appeared here. However, if this pedestrian was previously occluded by a parked vehicle, or had been moving towards the road, the historical information can help the model understand that this appearance is not an isolated event. The same applies when a neighboring vehicle suddenly approaches.
The current distance is merely a result; only the lateral position changes over the past few seconds can help the system determine whether it is driving normally or continuously approaching its own lane. In other words, the current state can tell the intelligent driving model what the present is like, while the historical state can tell the model how this state was formed through changes. And this is precisely the information needed to predict the future.
02. Why Is Predicting the Future Inseparable from the Past?
What autonomous driving truly needs to do is not just interpret the current frame, but decide the next action based on the current road state. However, the current state itself does not contain sufficient information. For example, a vehicle is currently decelerating. It might be a brief deceleration, or it might have been decelerating continuously for several seconds; it might soon resume normal driving, or it might stop due to congestion ahead.
These situations might look very similar at a single instant, but their corresponding futures are completely different. Therefore, when predicting the future, autonomous driving requires not only the current spatial state but also knowledge of the direction in which the state is changing. This is also a crucial reason why temporal information is introduced into autonomous driving models.
The CVPR 2025 paper, "Bridging Past and Future: End-to-End Autonomous Driving with Historical Prediction and Planning," specifically discusses the relationship between historical information and future prediction and planning. This research enhances perception and motion planning through historical information and validates this temporal modeling approach in open-loop and closed-loop tests on nuScenes.
The study points out that traditional methods either lose historical information in motion planning or fail to align with the multi-step nature of planning; historical information needs to be injected separately at different stages of perception and planning according to the time-step structure of the planning. The logic behind this can be understood as: what happened in the past determines how the current state is formed; and the current state and its trend of change determine what might happen in the future. Therefore, for an autonomous driving model to remember the past does not mean treating the past as an isolated information repository, but rather transforming the historical states relevant to the current driving into a part of future prediction.
03. What Exactly Does Autonomous Driving Know More After Remembering the Past?
For intelligent driving systems, truly valuable historical information is not necessarily the more, the better. Retaining a massive amount of historical content irrelevant to the current driving will only increase the computational cost of the model processing the context and may also introduce information unrelated to the current decision. What truly needs to be retained is the historical information that can explain the current state and help judge the future. For instance, if a pedestrian was just occluded, the vehicle needs to know where they appeared previously; if the leading vehicle starts to decelerate, the system needs to determine whether this change is a transient state or has formed a continuous trend.
This means the memory required by autonomous driving is not a complete memory in the human sense, but rather closer to a temporal context related to the current driving task. In the second-generation VLA upgrade plan announced by XPeng officially in August 2026, Infini-VLA is described as a long-sequence architecture, retaining a 30-second historical context on-board, allowing previously occurred actions, positions, and scenes to continue participating in current judgments; meanwhile, X-Foresight predicts scene changes in the next 6 seconds based on changes in vehicle position, speed, and environment, with the on-board scheme using 6 seconds as the prediction window.
04. Final Words
The Frontier of Intelligent Driving believes that the fact that autonomous driving models begin to remember the past does not mean that the driving logic has changed, but rather that the models begin to process the temporal attributes of driving itself more comprehensively. When judging the next action of the vehicle ahead, human drivers do not make decisions based solely on the present instant, but will combine the recently observed changes in speed, direction, and motion to make judgments.
Driving appears to be about predicting the future, but the basis for predicting the future often comes from past state changes. For autonomous driving, what is truly needed is not to remember all of the past, but to retain the temporal information that can explain the current state and help judge the future.
#AutonomousDriving #AutonomousDrivingPrediction