Author|Fan Yating
Hesai has made a new move in embodied AI, this time related to data.
Recently, Hesai officially launched the 3D World Data Engine Kosmo, comprising the AI spatial camera Kosmo One and the cloud platform Kosmo World. The former captures real-world environments, while the latter converts them into editable, reusable 3D spatial data, serving robot training, world models, and content creation.
Looking back at previous strategic moves, this was not a sudden pivot. Hesai's three co-founders have jointly established the robotics company Sharpa, which operates independently to develop robots and dexterous hands; in April of this year, Hesai also showcased Kosmo for the first time at its Technology Open Day and announced directions for robotic power modules.
After communicating with the Hesai team, you will find that Kosmo's ambition extends beyond just robotics. Embodied training, film production, and spatial display are all included in its application scope. Running through these scenarios is one core objective: Capturing the real world and turning it into reusable 3D data assets.
Everyone is capturing actions; Hesai captures environments first
"Currently, what Physical AI lacks is not data, but real-world data." This is how Xiang Shaoqing, Co-founder and CTO of Hesai, explained the starting point of Kosmo.
In his judgment, standard 2D images and videos lack directly measured depth, pose, and spatial relationships; pure simulation data struggles to fully replicate complex physical laws; real-machine teleoperation is sufficiently realistic but costly and difficult to scale. High quality, large scale, and low cost have long constituted the "impossible triangle" of embodied data.
Kosmo's entry point differs accordingly.
When previously experiencing Mifeng MEgo, I needed to complete the entire process of folding clothes or wiping tables, recording "how a person completes a task." Kosmo focuses more on the environment itself—the structure of the room, the position of objects, and the spatial relationships between them.
One centers on the operational process; the other centers on the reusable 3D environment.
This also determines Kosmo's position in the Real-to-Sim-to-Real pipeline: it primarily handles the initial phase where real environments enter the simulation system, lowering the barrier for scene capture and reconstruction.
Scan a room, then rearrange it
In terms of appearance, the 1kg handheld camera Kosmo One looks like a relay baton, but it is not as simple as it appears.
The device integrates a high-spatial-resolution LiDAR, an 8K imaging system, a 1-inch CMOS sensor, and a high-precision Inertial Measurement Unit (IMU), performing microsecond-level synchronized capture of geometry, color, texture, and device pose. Combined with 3D Gaussian Splatting (3DGS) reconstruction and specialized image processing tuning, the official claim is millimeter-level geometric accuracy.
These configurations have distinct roles: LiDAR provides spatial measurement, the imaging system records visual details, and the IMU assists in motion and pose estimation. Its approach ensures that 3D reconstruction is based on measured geometry rather than merely generating a visually similar room.
The capture workflow is simplified as much as possible. Kosmo One offers instant scanning, smart guidance, and one-click resume features. Typical parameters provided by the official source are: scanning a 100-square-meter ordinary indoor space takes about 15 minutes. Cloud model generation speed, under similar hardware, dataset, and quality requirements, is up to approximately 5 times faster than comparable 3DGS reconstruction products.
In the live demo, what attracted me most was the post-capture editability. Objects in the reconstructed scene can be individually selected and moved; the spatial layout is no longer fixed.
For robot training, this means one capture can expand into multiple training conditions. Recent demos of GPT-6 Astra using Blender to create 3D content showed the model's ability to construct virtual scenes; Kosmo starts from real spaces, adding scene variations through reconstruction and editing.
Supporting this step is Hesai's self-developed Kosmo Spatial Core (KSC). According to reports, this system identifies independent objects through semantic segmentation and fills unscanned areas through generative capabilities; each reconstruction synchronously outputs aligned 3DGS, Mesh, and Depth data, used respectively for visual rendering, surface geometry, and distance representation.
For example, after capturing a kitchen, one can change the position of cups, the arrangement of tables and chairs, constructing different training scenarios. The same task of grabbing a cup or clearing a table can be trained repeatedly under different conditions, helping robots learn to adapt to environmental changes. This also reduces the cost of repeatedly setting up sites, recapturing, and modeling, allowing one set of real-scene data to support more training tasks.
A DQ store provided a practical sample. According to Sharpa, their robots use existing equipment to make Blizzard ice cream at a DQ store in Shanghai. A single order involves 55 consecutive operation steps, including opening cabinet doors, adding ingredients, stirring, and flipping paper cups.
In this project, training environment modeling, which originally required about 300 hours by a professional team, was replaced by 15 minutes of scanning and reconstruction; the robot task success rate in this deployment reached 99%, reducing the failure rate to one-tenth of the previous level.
Robots need training grounds; films need sets too
Film creation demonstrates another use case for the same capability set.
In the hands of robotics companies, 3D scenes are training environments; in the hands of creators, they can become materials for location scouting, shot pre-visualization, virtual production, and post-processing. Compared to a fixed-camera video, spatial reconstruction retains the flexibility to choose viewpoints and process scenes further.
Currently, Kosmo has been adopted by top global film companies, including RenderMan, the industrial-grade renderer under Pixar. On October 5, RenderMan published a tutorial on its official website, using Kosmo to scan a desk lamp and small balls in the Pixar campus, importing the data into Houdini, rendering with RenderMan, and providing a 120MB sample project.
Artist Dylan Sisson explained in his usage record that capture results can enter Houdini's Solaris environment via USD format, without needing to recreate the plaza as a traditional polygonal scene first. In Houdini, scan results participate in editing as point data, allowing adjustments to position, size, and transparency before stylization.
Another launched scenario comes from AutoNavi (Gaode Maps). Among the first batch of cooperation projects announced by both parties, "Chong Art Museum" and "Pianfeng Gallery" were completed with 3D reconstruction by Kosmo, entering AutoNavi's Street View "Flying Street View," allowing users to browse real spaces online. Hesai is responsible for capture, reconstruction, and spatial data generation, while AutoNavi provides maps, location services, and application entry points.
Thus, Kosmo addresses not just one type of customer demand: Robotics companies purchase training materials, film teams create digital assets, and merchants and space operators need online displays.
Devices and Data Services: Two Business Models
Regarding Kosmo's business model, the Hesai team provided more specific explanations. One category is offering integrated software-hardware products to industry clients; the other is providing scalable real-space data production services for world model and Physical AI enterprises. Clients can capture data themselves or purchase data production services.
Correspondingly, Hesai plans to open access to captured images, point clouds, IMU data, and reconstructed 3D assets, interfacing with training and creation platforms such as Isaac Sim, Isaac Lab, Houdini, and RenderMan. On the day of the launch event, Kosmo also signed strategic partnerships with Galaxy General Robotics and Yuanli Lingji.
Yao Tengzhou, Co-founder of Galaxy General Robotics, introduced that before entering a new scenario, 3D reconstruction helps teams understand the environment in advance, allowing some training, debugging, and validation to be completed in the laboratory, shortening the delivery process upon arrival at the client site.
Zhang Xiaoheng, Commercial Vice President of Sudu Technology, discussed industrial clients' concerns about data: enterprises may not be willing to hand over their own scenario data to model companies. He believes end customers can also use Kosmo to capture data, then combine it with general models to complete post-training tailored to their own business needs.
Hesai is seeking its next growth curve. Comparing with the evolution of large models, all intelligence is built upon better "compression." Compression of network data brings model intelligence; next, how to achieve higher compression of the real world will directly impact the progress of Physical AI. Hesai's past LiDAR technology and business were essentially a method of compressing the physical world. Today, it hopes to cut in from this angle, truly evolving into a Physical AI company and a good robotics company.