When it comes to Hello, many people's first thought is the shared bikes commonly seen on the streets, but it has already ventured into the Robotaxi field. Recently, Hello Robotaxi released the generative driving world model HelloWorld. A key highlight of this work is that R&D personnel can actively modify vehicle trajectories, traffic participant layouts, and weather conditions, enabling the model to generate corresponding multi-view driving observations and attempting to connect the generated environment with driving policies.
For autonomous driving R&D, this provides a technical path different from fixed data playback. Technicians can not only observe driving processes that have already occurred but also study how the same scene would change under different trajectories and traffic conditions. Of course, scene generation is not a brand-new capability. What makes HelloWorld more noteworthy is its attempt to integrate scene control, 7-camera synchronous generation, LiDAR point cloud generation, and driving policy interaction into a single system.
To what extent have these capabilities been realized, and what issues still need to be verified? Let's explore this together today.
01. World Models Are Not Just About Generating Videos; They Must Also Be Able to Control Scenes
Real-world road data records driving processes that have already occurred. Whether used for model training or testing, R&D personnel can find specific scenarios from it. However, if there is a need to alter vehicle trajectories or traffic participant layouts in the original records, additional data construction or simulation methods are required.
HelloWorld attempts to provide a generative solution. According to its public introduction, R&D personnel can describe target scenarios through natural language or directly edit vehicles, pedestrians, and their motion trajectories. For example, they can make a preceding vehicle that was originally driving normally decelerate and stop within the lane, or adjust the position and movement of pedestrians to construct a scenario of crossing the road. The model then generates corresponding driving observations based on these conditions. Weather and lighting can also serve as control conditions. Day-night changes, environments such as rain, snow, and fog, as well as scenarios like construction and dense traffic, can all be used to guide the generation.
Such capabilities can make data construction more closely aligned with specific R&D problems. If a team is researching driving policies for when a preceding vehicle suddenly stops, they can construct different cases around the preceding vehicle's trajectory, rather than relying entirely on how much similar data is collected from real-world roads.
However, two concepts must be distinguished. Being able to generate a specified scenario does not mean that real and reliable test samples have already been generated. The appearance of pedestrians in the frames does not mean their movement necessarily conforms to real-world traffic rules; nor does the preceding vehicle stopping according to a specified trajectory mean the entire scenario satisfies complete vehicle dynamics and traffic constraints.
Therefore, in terms of scene generation, HelloWorld currently demonstrates a controllable data generation capability. Whether the generated data can effectively supplement long-tail scenarios and improve downstream driving models still needs to be confirmed through subsequent training and evaluation.
02. Synchronous Generation of 7 Cameras: The Challenge Is Not Just About Realistic Images
For a driving world model, generating images is just the foundation. After a vehicle changes its motion trajectory, the system also needs to generate road and traffic observations that match the new perspectives; the same vehicle seen by different cameras should also maintain reasonable position, appearance, and motion relationships. HelloWorld adopts progressive training, gradually transforming visual and motion knowledge from general videos into controllable driving generation capabilities. The model scale is approximately 2 billion parameters, and conditions such as ego-vehicle pose, camera calibration, HD maps, and 3D bounding boxes are progressively introduced during training.
There are two types of control information that need to be distinguished here. The ego-vehicle pose is used to control the spatial position and attitude changes of the vehicle, and the model generates corresponding driving observations based on this; HD maps and 3D bounding boxes are used to constrain the road structure and the spatial layout of traffic participants. The two types of conditions work together, enabling the model to adjust the ego-vehicle route and the surrounding scene separately. For example, if the vehicle is required to change from going straight to turning left, the model needs to generate observations corresponding to the new trajectory, rather than simply shifting the road images in the original video to the left. For a 7-camera system, each perspective also needs to maintain geometric correspondence as much as possible.
In the multi-view evaluation provided by Hello, HelloWorld was compared with Cosmos Transfer 2.5. Under the setting where both parties provide the first frame, the trajectory rotation error of the generated video decreased from 5.534° to 2.027°, a reduction of 63.4%; the cross-camera geometric error decreased from 6.549 pixels to 4.180 pixels, a reduction of 36.2%. This indicates that, under the evaluation conditions, HelloWorld's response to specified trajectories and cross-camera geometric correspondence have improved. However, the trajectory rotation error measures the motion results estimated from the generated observations, not the actual vehicle control error; the cross-camera geometric metrics also do not represent complete 3D geometric accuracy.
In addition to visual generation, HelloWorld also supports generating corresponding LiDAR point clouds based on camera RGB observations, and processes valid returns, distance information, and point cloud structures. This enables it to further possess conditional LiDAR point cloud generation capabilities beyond multi-view video generation, providing new possibilities for multi-sensor data construction. However, whether the correspondence between vision and point clouds is accurate, whether the generated point clouds conform to the distance information, return characteristics, and point cloud structures of real sensors still needs to be verified separately. One cannot conclude that it has completely simulated the sensor system of a real vehicle simply because the system outputs images and point clouds simultaneously.
03. From Generated Scenes to Policy Interaction: How Far Is It from Reliable Simulation?
HelloWorld also attempts to connect the generated environment with driving policies. In public demonstrations, the driving policy Alpamayo 1.5 outputs the ego-vehicle trajectory based on current observations, and HelloWorld then generates subsequent frames based on the trajectory and previously generated observations for the policy to continue making judgments.
Compared to fixed trajectory playback, this method can generate subsequent observations corresponding to the new trajectory given by the policy. R&D personnel therefore have the opportunity to study the changes of the same scene under different driving choices, without being entirely limited to the single path in the original records. However, the closed-loop here refers to the interaction process between the driving policy and the generated observations, not that the system has already established a fully verified vehicle dynamics simulation environment. During the continuous generation process, the world model will continuously read its own previously generated history.
Early generation errors may affect subsequent observations, and the new trajectory selected by the policy may also cause the scene to gradually deviate from the range covered by real records. To address this issue, HelloWorld adopts chunked causal generation, allowing the model to generate subsequent observations segment by segment, and improves the model's ability to handle imperfect contexts during continuous operation through training adaptation with perturbed history and its own generated history. The model also uses few-step distillation to compress a 20-step teacher model into a 4-step student model, reducing the computational cost of repeated generation.
In the provided efficiency tests, the model uses 8 GPUs to generate under the conditions of 720p resolution, 7 cameras, 61 frames, and providing the first frame. The sampling time decreased from 357.7 seconds to 77.8 seconds, an acceleration of 4.60 times; the end-to-end time, including video exporting and other processes, decreased from 426.7 seconds to 143.2 seconds, an acceleration of 2.98 times. These results indicate that reducing generation steps can lower inference overhead under specific configurations, but minute-level end-to-end time does not represent real-time generation. Nor can it independently prove that the model is already capable of stably simulating long-duration driving processes.
More importantly, policy performance in the generated environment cannot be directly equated to driving capabilities on real roads. Vehicle dynamics, the response of traffic participants to the ego-vehicle's actions, and error accumulation during long-term operation all require further research. Whether the generated environment can be used for reliable safety evaluation also requires systematic verification.
04. Final Words
Hello's HelloWorld attempts to integrate controllable scene generation, 7-camera synchronous generation, LiDAR point cloud generation, and driving policy interaction into a single system. This allows world models to move further from merely generating driving images towards data construction and interactive simulation for autonomous driving R&D. However, generation capability does not equal reliable testing capability.
Whether generated data can improve downstream models, whether policy interaction can remain stable during long-term operation, and whether simulation results can effectively reflect real driving performance still require more systematic evaluation. For the autonomous driving industry, the value of world models lies not only in generating more scenarios but also in whether these scenarios can be transformed into repeatable, evaluable, and truly useful testing resources for R&D.