
Spatial AI is artificial intelligence that perceives the three-dimensional world, builds a model of where things are, and reasons about how they relate to each other and to the machine itself. It is what lets a robot find a cup on a cluttered table, a drone hold its position under a bridge, or an AR headset pin a virtual screen to a real wall.
The term has a second meaning too. In mapping and location analytics, "spatial AI" means machine learning on geographic data. This guide covers both, then goes deeper into the 3D sense: the tasks it is made of, how it differs from physical and embodied AI, and why training data is the hardest part.
The 3D meaning of spatial AI grew out of robotics. Andrew Davison of Imperial College London described it in his 2018 paper FutureMapping as the evolution of SLAM (simultaneous localization and mapping) into a general geometric and semantic perception capability for robots and other devices that act in the world. In plain terms, the system doesn't just see pixels. It knows that the object is a chair, that it is 1.2 meters away, that it sits on the floor and that the robot can walk around it.
A 2026 survey on spatial AI agents puts it in one line: spatial intelligence is "the ability to perceive 3D structure, reason about object relationships, and act under physical constraints." Perception, reasoning and action. Remove any one and you are back to ordinary computer vision or ordinary planning.
Search for spatial AI and half the results talk about maps. Location analytics vendors use the term for AI applied to geographic data: predicting foot traffic for a store site, modeling flood risk by postcode, segmenting customers by neighborhood. That field is usually called GeoAI, and its data lives in latitude, longitude and map layers.
The robotics meaning works at a different scale and on different data. It deals with centimeters instead of kilometers, with depth images and point clouds instead of map layers, and with a machine that has to act in the space in real time. The two fields share techniques (both care about coordinates and geometry) but they solve different problems for different buyers. The rest of this article is about the 3D, machine-perception sense.
Large language models learned from text, and text describes space loosely. "The keys are next to the laptop" says nothing about distance, orientation or whether the keys will fall off the table if the laptop moves. A model trained on billions of such sentences can talk about space fluently without holding any internal 3D geometry.
Vision-language models get closer because they see images, but an image is still a flat projection. Ask one to estimate how far apart two objects are, or what a scene looks like from the other side, and the answers are frequently wrong. Benchmarks such as 3DSRBench and ViewSpatial-Bench show leading models falling well short of humans on orientation and viewpoint questions. The arXiv survey makes a related argument about language models: success in symbolic domains does not carry over easily to the physical world, where grounding in space matters. Fei-Fei Li has framed spatial intelligence as AI's next frontier, and her company World Labs launched Marble, a model that turns text, images or video into editable 3D worlds, in November 2025.
Spatial AI is not one model. It is a stack of perception tasks, each producing a different piece of the 3D picture:
| Task | What it outputs | Where it is needed |
|---|---|---|
| Depth estimation | Distance to every pixel, from stereo cameras or a single image | Obstacle avoidance, AR occlusion, grasping |
| 3D object detection | 3D boxes with position, size and heading | Autonomous vehicles, warehouse robots, drones |
| 6D pose estimation | An object's exact position and rotation | Robotic picking, assembly, bin picking |
| SLAM | A map of the environment and the device's position in it | Mobile robots, AR/XR headsets, robot vacuums |
| 3D semantic segmentation | A class label for every point or voxel in a scene | Mapping, inspection, scene understanding |
| Scene graphs | Objects plus relations: on, inside, next to, supports | Task planning, instruction following |
These tasks feed each other. SLAM gives the frame of reference, detection and pose tell the system what sits where, and the scene graph turns that into something a planner can reason with ("the cup is on the tray, so lift the tray"). A failure low in the stack, such as a depth error of a few centimeters, shows up higher up as a missed grasp. Our overview of computer vision in robotics shows how these pieces come together on real robots.
The three terms overlap, people often use them interchangeably, and there is no standard taxonomy. Davison himself framed spatial AI for embodied devices. Still, one useful way to split them in common usage is by layer:
| Spatial AI | Physical AI | Embodied AI | |
|---|---|---|---|
| Core question | Where is everything, and how does it relate? | How do I act on the physical world? | How does an agent learn through its own body? |
| Main output | A 3D understanding of the scene | Actions with physical consequences | Behavior learned from interaction |
| Needs a body? | No (a phone or headset is enough) | Yes, a machine that acts | Yes, the body is part of the learning |
| Typical examples | SLAM in AR glasses, 3D mapping, pose estimation | Humanoids, autonomous trucks, factory arms | Robots learning to walk or grasp by trial |
The simplest way to hold it: spatial AI is the understanding, physical AI is the acting, and embodied AI is a way of learning that ties the two to a specific body. A robot needs all three, but an AR headset only needs the first. For the full picture of the other two, see our guides on what physical AI is and embodied AI vs physical AI.
Most spatial AI already in use is quiet and practical. Most robot vacuums that map your flat and remember where the kitchen is run SLAM. Phone AR apps that keep a virtual sofa on the floor while you walk around it rely on the same kind of visual-inertial tracking.
In industry the stakes rise. Warehouse robots use 3D detection and pose estimation to pick mixed items from bins. Many autonomous vehicles fuse camera, LiDAR and radar into a 3D model of the road, each sensor covering the others' blind spots. Drones inspect bridges, power lines and wind turbines by building 3D models of the structure and flagging changes between flights. Digital twins of factories and buildings depend on the same capability to keep the virtual copy aligned with the real one.
The newest frontier is robots that take spoken instructions. "Put the red box on the second shelf" requires the system to ground every word in 3D: which box, which shelf, counted from where. That grounding is spatial AI, and it is where much of the current research effort goes.
World models are one of the main routes researchers are taking toward spatial intelligence. Instead of only labeling what is in a scene, a world model learns to predict how the scene will look and change: what happens if the robot pushes the box, what lies behind the door. Generative 3D models like World Labs' Marble come at it from the other side, producing whole navigable environments.
Neither approach replaces the perception stack above. A world model still needs accurate geometry to predict from, and its predictions are only useful if they respect physics. We looked at that tension in world models vs 3D simulation.
Every task in the table above needs ground truth in 3D. A depth model needs the true distance for each pixel. A pose model needs the exact rotation of each object, to the degree. A 3D detector needs a box drawn around every car, pallet or person, in space, frame after frame.
Real data makes this hard in three ways. Some labels are close to impossible for a human to draw: annotating a 6D pose by eye to sub-degree accuracy is practically out of reach, and painting a correct depth value onto a million pixels is not feasible by hand. Some require expensive sensors, such as LiDAR rigs or motion capture, just to measure the truth. And the scenes that break models are rare by nature, like a transparent bottle under glare or a forklift half hidden by shelving, so collecting enough of them takes months.
Synthetic 3D data removes most of that friction. When a scene is rendered from 3D assets, the renderer already knows where every object is, how far it is from the camera, how it is rotated and which pixels belong to it. Depth maps, poses, segmentation masks and point clouds come out with every image, geometrically exact within the simulated scene and with no manual labeling errors. Rare conditions become a parameter: change the lighting, move the occluder, swap the material, render another thousand frames.
The catch is the gap between rendered and real images. Models trained only on synthetic data can stumble on real sensors, which is why teams mix in real data and vary textures, lighting and camera properties widely during generation. Our article on the sim-to-real gap covers how teams close it in practice.
Start from the task, not the trend. Decide which output your system actually needs (depth, 3D boxes, 6D pose, a map) and how accurate it must be, then look at what labeled data exists for your objects and environments. For most industrial teams, that last question decides the timeline: public datasets rarely contain your parts, your warehouse or your lighting.
That is the gap Vivid 3D fills for robotics and physical AI teams. We build 3D scenes from your assets and generate synthetic datasets with depth, pose, segmentation and point cloud labels, including the rare cases that real data misses. If you are planning a spatial AI project, see how we support robotics teams or book a demo to talk through your use case.
Spatial AI understands 3D space: where objects are, how big they are and how they relate. Physical AI uses that understanding to act in the real world through a robot, vehicle or machine. Spatial AI can run on a phone or headset with no body at all, while physical AI always controls something that moves.
No, though they are closely linked. Spatial computing is the broader idea of computers that blend digital content with physical space, as in AR and VR headsets. Spatial AI is one of the capabilities that makes it work, by mapping the room and tracking the device's position in it.
Spatial AI models train on images paired with 3D ground truth: depth maps, point clouds, 3D bounding boxes, 6D object poses and segmentation masks. This data comes from LiDAR and depth sensors, motion capture, manual annotation and, increasingly, synthetic 3D scenes where every label is generated automatically.
