On August 11, XPENG's foundation model team submitted a research paper named XCoT-VLA to arXiv. Instead of generating a longer Chain-of-Thought (CoT) for intelligent driving, it compresses the driving reasoning representation into 2 to 6 executable semantic tokens. This seems contrary to our common perception: since autonomous driving requires reasoning, why shorten the thinking process?
01. Why Can't Intelligent Driving Think Slowly Like a Chatbot?
For general language tasks, a longer reasoning process incurs additional computational costs and response latency. As users, this delay does not have a significant impact; however, for autonomous driving, such latency can actually cause danger. With every step the car moves forward, the surrounding environment may change. The system needs to quickly re-evaluate when the leading vehicle starts to decelerate, a neighboring vehicle suddenly changes lanes, traffic lights change, or even when a pedestrian appears from an occluded area. This means the thinking required for intelligent driving is not exactly the same as the thinking of a chatbot when answering questions.
A chatbot can use lengthy natural language reasoning to explain a problem from conditions and causes all the way to the conclusion. Autonomous driving certainly also requires complex judgments, but the vehicle ultimately does not need to speak the answer; rather, it needs to continue generating the next segment of the driving trajectory. This raises a very practical issue: if the model spends a large amount of computational resources generating a long natural language explanation, these words themselves cannot directly correspond to the vehicle's future continuous trajectory. Traditional Chain-of-Thought unfolds complex problems step by step through continuous natural language reasoning.
For general large models, this approach helps improve reasoning capabilities for complex tasks. However, in driving scenarios, open-ended language output has two inherent problems: first, the output length is uncontrollable; second, natural language expressions are not naturally aligned with the vehicle's future continuous trajectory, requiring further translation of semantic judgments into trajectories. For example, when a vehicle approaches a traffic light. The key judgments the model truly needs to form might just be a few pieces of information directly related to driving behaviors, such as red light, maintain current lane, decelerate, and stop. If the model first generates a large paragraph of natural language explaining that there is a red light ahead, so the vehicle should gradually reduce speed and stop before the stop line, it is easier for humans to understand. But for the subsequent trajectory generation model, this expression is not the most compact input format.
The XCoT proposed by XPENG, which stands for Executable Chain-of-Thought, solves exactly this problem. Instead of making the model explain the driving process in more detail, it translates driving reasoning into more compact and action-oriented semantic tokens. Traditional natural language reasoning often requires 40 to 80 tokens, whereas XCoT requires only 2 to 6. Decoding each token on an H100 takes about 3.25 milliseconds, and even in the worst-case estimate, it is below the 83.3-millisecond budget allowed by the vehicle's 12Hz control frequency. In this way, the model does not need to spend a large number of tokens describing why it is doing something, but can directly form intermediate semantics that can influence subsequent driving behaviors. Therefore, what XCoT truly pursues is not to make intelligent driving think less, but to make the thinking results closer to what the vehicle needs to do next.
02. First, Figure Out Exactly What Needs to Be Thought About
Speaking of this, a misunderstanding can easily arise. Does XCoT add an extra decision-making module specifically responsible for thinking into the original intelligent driving system? Not exactly. More accurately, XCoT provides an intermediate semantic representation layer that connects multimodal scene understanding with vehicle trajectory generation. The vehicle first needs to understand the current scene from multimodal inputs such as vision, while combining ego-vehicle status, historical trajectories, and navigation information to form an overall judgment of the current driving situation. For example, an intersection appears ahead, the current light is red, there are vehicles in the left lane, and the navigation requires going straight.
These pieces of information themselves are not yet the final trajectory the vehicle needs to execute. The system needs to make further judgments: should it maintain the current lane or change lanes? Does it need to decelerate? Does it need to yield? Does it need to stop? What XCoT does is express this layer of judgment using a set of compact semantic tokens. For instance, in a specific scenario, the model might form a semantic combination similar to maintain lane + decelerate + stop at red light. They express the current driving state and behavioral intentions, rather than the final steering wheel angle, acceleration values, or vehicle trajectory.
This point is crucial because the motion of a real vehicle is continuous. Even if the system has already judged that deceleration is needed, it still needs to combine information such as current vehicle speed, road curvature, distance to the leading vehicle, and vehicle dynamics to further calculate exactly how much to decelerate, when to start decelerating, and what trajectory to follow over the next period. The XCoT-VLA in the paper keeps these XCoT tokens in the model context and uses them as conditional information for subsequent trajectory generation. The model ultimately still needs to generate a continuous vehicle trajectory, rather than directly treating a token like decelerate as a vehicle control command. Therefore, what XCoT changes is not a simple sequence of decision-making first and planning later, but adds a more explicit action semantic expression between multimodal scene understanding and continuous trajectory generation. This also explains why the paper did not call it a general driving decision token.
It emphasizes executability, meaning these reasoning results need to serve subsequent actions, rather than simply allowing humans to understand why the model made such a judgment. From this perspective, thinking clearly first does not mean making the car form a complete language thinking process in its mind like a human, but rather having the model express the key driving intentions corresponding to the current scene before actually generating the trajectory. This is actually an increasingly noteworthy issue since Vision-Language-Action (VLA) models entered autonomous driving. VLA models do not just enable cars to understand more things; they also attempt to connect visual understanding, semantic reasoning, and action generation. The difficulty lies precisely in this intermediate step: what is seen, what kind of judgment should be formed, and how to translate this judgment into behaviors that the vehicle can execute? XCoT provides a relatively direct connection method.
03. Why Is Thinking Clearly Not Enough; It Must Also Be Timely?
The reasoning in autonomous driving must not only be correct but also timely. The vehicle will not wait for the model to slowly finish writing out an entire set of reasoning processes. Suppose the vehicle is traveling at a high speed and a risk that needs to be handled suddenly appears ahead; the system must complete environment understanding, behavior judgment, and trajectory updating within a limited computation cycle. If the reasoning process itself becomes too long, even if the final judgment is correct, it may lose its practical significance due to a response that is too slow. This is also a key reason why XCoT compresses the reasoning representation into typically 2 to 6 executable semantic tokens. The paper points out that open-ended natural language CoT has problems such as high decoding costs and uncontrollable output length in real-time driving control, while XCoT reduces this overhead through fixed and compact action semantics.
In the system design of the paper, XCoT and subsequent trajectory queries jointly participate in multimodal attention computation, and the trajectory generation part ultimately outputs the future vehicle motion. The 2 to 6 tokens here do not mean that a single driving decision requires only 2 to 6 tokens, nor does it mean that the entire reasoning process of autonomous driving is compressed into a few words. A more accurate understanding is that the model uses a very short piece of structured semantics to express the most critical and directly action-oriented information in the current driving reasoning. This expression method has another advantage: it is easier to form stable driving semantics.
Natural language can be expressed in many ways. Expressions such as there is a red light ahead, the vehicle should gradually decelerate and stop before the stop line, need to maintain the current lane and wait for the traffic light to change, and temporarily do not continue driving forward have similar meanings to humans, but may correspond to completely different token sequences for the model. By adopting more standardized action semantics, XCoT can reduce the uncertainty brought by such open-ended language expressions, making intermediate reasoning results serve subsequent trajectory prediction more stably. The open-loop planning experiments in the paper also show relatively obvious changes. On the general distribution test set, the longitudinal Average Displacement Error (ADE) decreased from 1.645 to 1.323; in the lane-changing scenario, the lateral Final Displacement Error (FDE) decreased from 1.616 to 0.648. These are experimental results in the paper and cannot be directly equated to the actual road performance of mass-produced vehicles.
Furthermore, these results belong to open-loop planning experiments, and the paper itself points out that interactions and traffic rule perception in real closed-loop driving still need further verification. But it at least illustrates one thing: reasoning in autonomous driving does not necessarily need to become longer and longer; instead, it may need to get closer and closer to action. Past intelligent driving systems emphasized more on what is seen, followed by how to drive. However, when end-to-end models and VLA gradually connect perception, reasoning, and action generation further, this intermediate layer of judgment begins to become more important. A car seeing a red light does not mean the car already knows how to handle it; seeing a car next to it does not mean the system has already completed the judgment on the timing for a lane change.
What is truly valuable is translating these environmental information pieces into driving intentions that can influence the next action. Therefore, XPENG is not adding a talking Chain-of-Thought to intelligent driving, but attempting to answer a more practical question: when autonomous driving begins to require reasoning, in what form should this reasoning exist? The answer given by XCoT is to make it shorter, more explicit, and closer to action.
#AutonomousDriving #XPENG #XCoT