Author | Su Jialing
A smooth lane change in a simulator may not withstand the test of real-world roads. The ego vehicle steers, enters the adjacent lane, and the rear vehicle continues along its original trajectory and speed; the entire process ends smoothly. However, on real roads, vehicles in the adjacent lane might decelerate to create space or accelerate to close the gap. The ego vehicle must then decide whether to continue merging or delay the lane change based on the other driver's reaction.
The difficulty of driving lies precisely in this back-and-forth interaction.
In simulators relying on fixed trajectory playback, surrounding vehicles resemble "puppets" moving according to a script: even if the ego vehicle changes its action, they still proceed along predetermined paths. When simulated vehicles do not react to you, no matter how realistic the visuals, it may train incorrect driving capabilities.
A seemingly successful operation can mask risks in decision-making. The more the model practices, the more it may trust this false sense of security. BehaviorWorldGen, released by Afari Technology's World Model Team, attempts to add behavioral and responsive capabilities to these simulation "puppets": developers can specify that a certain vehicle takes a specific action at a certain moment, while surrounding vehicles generate reasonable reactions accordingly.
Autonomous driving simulation is not just about making the virtual world look like reality; it also requires traffic participants in the virtual world to make reasonable responses, verifying the model's adaptability to real traffic conditions.
01 Simulated Vehicles Cannot Just Follow a Script
Autonomous driving lacks neither daily driving records nor moments that expose model weaknesses—namely, long-tail scenarios. In the past, autonomous driving companies competed on road test mileage, using a sufficiently large denominator to encounter more long-tail scenarios. Waymo found that among approximately 6.391 million miles of driving logs, after automatic screening and manual review, data meeting their target long-tail scenario standards accounted for only about 0.03% of the original mileage.
Even with mileage scales as large as Waymo's, long-tail scenarios are difficult to exhaust through mere "chance encounters."
Example of long-tail scenarios from Waymo's End-to-End Driving Open Dataset (WOD-E2E). Finding cases is only the beginning. Real roads rarely cooperate with R&D to repeatedly reproduce these conditions. Simulators provide the opportunity to proactively "set questions": adjusting scenarios, repeating tests, and allowing different versions to be examined against the same problem. This March, Xpeng's X-World simulation scenario library increased from about 30,000 a year ago to over 500,000, with daily simulation test mileage equivalent to 30 million kilometers of real-vehicle testing.
But the issue is that generating a world does not equate to generating a realistically functioning traffic world. Waymo's VP of Vehicle Software shared that open-loop simulation replaying recorded data is insufficient because it is like walking into a video playback where surrounding traffic moves but completely ignores your behavior. Driving decisions are constantly adjusted through interactions with other road participants.
The controllability of traffic behavior has become a critical capability for simulation.
Xpeng's X-World provides a set of conditional control interfaces, incorporating ego vehicle actions, surrounding traffic participants, road elements, and camera parameters into the control scope to achieve controllable multi-view generation. Afari Technology's BehaviorWorldGen framework focuses on the control and generation of traffic behaviors, bridging two aspects:
First, the behavior of road participants is controllable. Simply tagging a trajectory with a "lane change" label is not enough; R&D needs to further determine: which vehicle changes lanes, and when. A core component of BehaviorWorldGen, BehaviorFlow, is a meta-action conditioned traffic flow model. It converts behaviors such as lane keeping, lane changing, turning, and U-turns into explicit frame-level meta-action commands, and then generates vehicle motion accordingly. The key to this design is giving behavior control a clear temporal anchor.
Frame-level meta-action control aligns behavior commands with frame-by-frame state transitions, allowing behavior injection into specified vehicles at specified moments. For example, assuming the front vehicle cuts in at the 3rd second, one can examine how the driving model responds. With this layer of control, difficult scenarios need not rely entirely on random generation. Developers can arrange vehicle actions and their timing around specific issues requiring verification.
Second, response correlation allows one vehicle's action to alter the entire traffic situation. BehaviorFlow decomposes the generation process into three parts:
- Lifecycle factors determine which vehicles enter, remain, or leave the scene. Meta-action factors determine what behaviors they take. State factors then generate specific physical states under the constraints of the former two.
The three factors are decoded sequentially: first determining which cars participate in the interaction, then deciding what they will do, and finally generating specific states. The key to interactive capability lies in the final step, where state factors jointly generate across all active vehicles. Injecting one behavior control simultaneously alters the reactions of the controlled vehicle and surrounding vehicles, achieving multi-vehicle interaction consistency.
An intersection reveals this distinction clearly. The ego vehicle prepares to turn left, while an oncoming vehicle prepares to go straight. From the same starting point and direction, two different passing orders can be generated: the oncoming vehicle decelerates and waits, allowing the ego vehicle to turn first; or the ego vehicle yields, allowing the oncoming vehicle to pass first.
From the same initial scenario, multiple interaction outcomes are generated via meta-action constraints. Developers set the target action, while the responses of unconstrained vehicles are generated by the model. New vehicle states form new traffic situations, which then serve as the basis for subsequent generations.
Who goes first, who waits, and how another vehicle acts next—behavior control thus extends from one vehicle's action to coordination among multiple vehicles, evolving continuously in subsequent generations.
BehaviorWorldGen attempts to connect two capabilities: "making target behaviors occur on demand" and "allowing surrounding traffic to change accordingly." Once connected, simulated vehicles have the chance to become training opponents for driving models.
02 Targeted Simulation for Model Weaknesses
Enabling simulated vehicles to participate in interactions solves the problem of "whether we can practice." However, if training data still largely repeats scenarios the model has already mastered, new investments may not address current shortcomings. For simulation to enter the training closed loop, model failures must become the basis for the next round of data generation. BehaviorWorldGen's approach is to generate targeted training samples for failure cases. The action model outputs an initial trajectory based on current observations. Scenario-level diagnostics subsequently identify conditions such as collisions, unsafe following distances, and yielding failures, converting these issues into target meta-action prompts that BehaviorFlow can receive to construct corresponding traffic interactions.
Subsequently, BehaviorFlow jointly generates vehicle states under behavioral constraints. The world simulator renders scenes into multi-view RGB and LiDAR observations, pairing them with corrected interaction-aware trajectories and adding them to the training data. A failure may occur at specific distances, speeds, and action timings. Refining control to specific vehicles and specific moments ensures the generated interactions are closer to what the model needs to practice. The effect is concentrated in the most difficult scenarios. In the filtered hardest scenario intervals, the original model's PDMS (Predictive Driver Model Score) was <0.15, near collapse. After training with data generated by BehaviorWorldGen, the score recovered to 34.8, restoring planning performance.
From model errors to diagnosing failure conditions, and then to generating corresponding experiences, previously dispersed links now form a closed loop. Beyond interaction, BehaviorWorldGen also supports weather style transfer, continuous fog control, rare object insertion, and trajectory editing, expanding the same failure scenario into variants with different visibility and obstacle conditions.
Beyond controllability, it must also be fast and high-fidelity.
The richer the training conditions, the more generation tasks arise. If scene production is too slow, targeted training cannot be conducted frequently. In comparative experiments disclosed by Afari Technology, adopting causal generation for the action-conditioned world model compressed per-frame generation time from approximately 20 seconds in bidirectional models to 0.3 seconds, nearly 70 times faster. Generation time for a 93-frame video shortened from about half an hour to approximately 28 seconds.
The same time budget can support more scenario variants and more frequent training adjustments.
Speedup did not sacrifice image quality. Measured by FID, the causal model achieved 4.36, superior to the bidirectional model's 6.72. The framework also supports approximately 30-second long-horizon autoregressive generation, allowing observation of continuous changes after an action. When driving models attempt different actions, the virtual environment needs to accommodate new routes.
Scenes reconstructed from real road records maintain rendering quality better in areas with sufficient coverage of original observations. Once the ego vehicle detours for avoidance, new viewpoints may exceed the coverage range of existing data. At this point, model performance suffers from missing or distorted scenes.
BehaviorWorldGen uses 3D Gaussian Splatting for scene extrapolation to expand the usable range of reconstructed scenes, then employs LiDAR supervision to constrain the geometric structure of extrapolated regions, ensuring the model obtains usable subsequent observations even after rerouting. From failure diagnosis to targeted generation, and then to generation acceleration and spatial expansion, BehaviorWorldGen attempts to shorten the distance between "discovering what the model doesn't know" and "providing effective practice," giving long-tail scenarios—which are hard to wait for, collect, and repeat—the possibility of proactive orchestration.
03 Autonomous Driving Simulation: More Than Just Volume, It Must Teach Effectively
Currently, autonomous driving model architectures and implementations exhibit a parallel multi-route landscape. During the stage where technical routes such as VLA, end-to-end, and world models have not yet converged, Afari Technology needs to solve: making controllable interactions adaptable to different combinations rather than limited to a single system.
The BehaviorWorldGen framework chooses structured trajectories as the common interface to connect upstream and downstream components. Upstream, it can integrate VLA models and other end-to-end planners; downstream, it can connect to generative world models and reconstruction-based simulators based on 3D Gaussian Splatting. The model architecture adopted upstream does not directly dictate the simulator used downstream; similarly, how the downstream generates visuals is not bound to a specific driving strategy. Behavior generation and scene presentation can be improved separately, linked through the BehaviorWorldGen framework.
Thus, interactive generation capability is not limited to a single technical route. The research team tested the training effect of new data on three action models. After adding 16,000 generated samples to the original 85,000 NAVSIM training data points, ChainFlow-VLA (Vision-Language-Action model)'s PDMS improved from 93.1 to 93.3, ReCogDrive (Vision-Language-Action model) improved from 86.5 to 87.3, and DiffusionDrive (end-to-end planner) improved from 87.7 to 88.6. All three architectures showed improvement, indicating that the utility of generated data is not confined to a single model.
The value of compatibility thus falls onto R&D accumulation.
The more technical combinations a method serves, the greater the chance it transforms from a dedicated tool for single experiments into a reusable data production capability. Teams do not need to rebuild the complete pipeline each time but can continuously improve around specific modules. As the weight of synthetic data in training increases, the importance of this capability rises.
According to Zosi Auto Research's "2025 China Intelligent Assisted Driving Data Closed Loop Report," the proportion of synthetic data in autonomous driving training data increased from about 20%-30% in 2023 to 40%-50% in 2024, exceeding 50% in 2025. When more driving experience comes from generation, the quality of simulation data more directly impacts model capability. Rich scenarios and realistic visuals are merely the foundation.
As a virtual training environment for AI to learn real-world laws, world simulators are participating more deeply in training and validation closed loops, with tighter connections between scene generation, training, and evaluation. Waymo sets up a Critic alongside Driver and Simulator to analyze driving behavior in real roads and simulations, providing signals for training and assessment. Road experience, after filtering, annotation, and model updates, must undergo simulation and safety verification to judge whether improvements hold.
BehaviorWorldGen's entry point is more specific: supplementing training data for failure scenarios with controllable interactions, and adapting to different models and simulators through a common interface, integrating this capability into the training and validation closed loop. Problems encountered on the road can become the starting point for diagnosis and generation; the validity of generated data continues to be tested by training and validation; remaining failures continue to drive diagnosis and sample supplementation.
This closed loop supplements training experience but cannot replace real-road validation. As the weight of synthetic data grows heavier, beyond generating scenarios, continuous and verifiable model progress represents a smokeless competition in autonomous driving simulation.