Computer vision in retail means cameras and AI models that read the store for you: which shelves are empty, which items went into a basket, which products a shopper photographed on their phone. Shelf monitoring, checkout automation and loss prevention are the uses with real deployments today, across thousands of stores.
This guide covers eight use cases with named examples, a comparison table, and the part most retail AI articles skip: why these models break once they leave the pilot store, and where the training data that fixes them comes from.
Computer vision in retail is the use of cameras and machine learning models to detect, recognize and track products, shelves and people in stores and warehouses. The models turn video and photos into structured data, such as an out-of-stock alert, a list of items in a cart or a planogram compliance score, that store teams and systems act on.
The cameras vary a lot. Morrisons uses tiny shelf-edge cameras. Checkout-free stores hang dense arrays from the ceiling. Some planogram audits run on nothing more than an associate's phone. The underlying tasks are the same few computer vision problems: object detection, product classification, segmentation and tracking.
A planogram is the plan for which product goes where on a shelf. Checking it by hand means a person walking aisles with a printout, so audits happen rarely and drift goes unnoticed for weeks.
Vision systems photograph the shelf, detect every product facing and compare the result to the digital planogram. The largest documented deployment runs across more than 7,000 7-Eleven stores in Taiwan. A December 2025 paper in Scientific Reports describes how it works: the system stitches several photos into one "virtual shelf" because narrow aisles rarely let a single camera capture a full shelf.
Empty shelves cost sales the moment they appear, and staff usually find them late. Morrisons started trialing Focal Systems shelf-edge cameras in one store in 2022. Focal says the full rollout put more than 200,000 cameras into 498 Morrisons supermarkets in about eight months. The cameras photograph shelves through the day, flag gaps to staff and trigger reorders when stock runs low.
Robots do the same job from a different angle. Simbe's Tally robot drives the aisles and scans shelves, which suits stores that don't want hundreds of fixed cameras.
Amazon's Just Walk Out is the best-known example: ceiling cameras and shelf sensors track what each shopper picks up, and the payment happens when they leave. Amazon says the system runs in more than 375 stores across five countries, mostly smaller venues like stadiums, airports and campuses.
The history is less tidy than the marketing. In April 2024 Amazon pulled Just Walk Out from its US Amazon Fresh grocery stores and switched them to Dash Carts, which scan items as shoppers add them. Full-store camera tracking is easier to run in a small store with a few hundred items than in a supermarket with tens of thousands.
Self-checkout moved scanning from staff to customers, and shrink followed. Cameras mounted above the scanner watch each item and compare it with what was actually scanned. When a shopper scans a cheap item and bags an expensive one, or skips the scan entirely, the system flags the transaction for an attendant. Several UK and European grocers run this kind of system. Its payback is easy to measure, because every flagged item is a sale that would otherwise have been lost.
Overhead cameras count people and track anonymous paths through the store. The output is a heat map: where shoppers linger, which aisles they skip, how long the checkout queue gets at 5 p.m. Retailers use it to test layouts, staff the floor and measure whether an endcap display actually stops anyone. Good systems track bodies as anonymous points and never identify faces.
Visual search lets a shopper photograph a product and find it, or something like it, in the catalog. Target added Pinterest Lens to its app in 2017 for this. In B2B the same technology matches spare parts and fittings: a technician takes one photo, and the model finds the exact variant among hundreds of near-identical SKUs.
That last example shows what makes retail product recognition hard. The model has to distinguish a 500 ml bottle from a 1 L bottle of the same brand, or a satin finish from a matte one.
Face and body tracking places makeup, glasses or shoes on the shopper's live camera image. L'Oréal bought ModiFace in 2018 and uses it across brands like Maybelline and Lancôme. Virtual try-on sits close to augmented reality in ecommerce: both need an accurate 3D model of the product and a model that understands the scene it's placed into.
Grocery chains use cameras to grade fresh produce before it reaches the shelf. Walmart's Eden system, first reported in 2018, compares scans of incoming produce with a library of about a million photos tied to USDA quality standards. Distribution center staff use it to record defects and pull produce that fails the spec. Walmart reported early savings of $86 million from reduced waste.
| Use case | Camera setup | Main CV task | Training data difficulty |
|---|---|---|---|
| Shelf monitoring and planograms | Shelf-edge cameras, robots or phones | Dense object detection, SKU classification | High: thousands of SKUs, constant packaging changes |
| Out-of-stock detection | Shelf-edge or ceiling cameras | Gap detection, product detection | Medium |
| Cashierless checkout | Dense ceiling arrays plus shelf sensors | Multi-person tracking, item recognition in hand | Very high: occlusion, many people at once |
| Self-checkout loss prevention | Camera above each scanner | Item recognition, action recognition | Medium to high |
| Foot traffic and heat maps | Overhead cameras | Person detection and tracking | Low: pretrained models work well |
| Visual search | Shopper's phone | Image retrieval, fine-grained classification | High: near-identical variants |
| Virtual try-on | Phone or smart mirror | Face and body tracking, 3D rendering | Medium |
| Produce inspection | Fixed cameras at receiving | Defect classification | Medium: needs labeled defect examples |
A shelf detection model that scores well in the pilot store often loses accuracy in store number fifty. The causes are specific to retail.
The catalog never stops changing. A large supermarket carries tens of thousands of SKUs, and suppliers redesign packaging, add seasonal editions and launch new flavors every month. Each change can turn a known product into an unknown one. A model trained once and left alone degrades on a schedule.
Variants look almost identical. Same brand, same colors, different size or flavor. Fine-grained differences like "zero sugar" printed in small type are hard to learn from a few dozen photos.
Shelves are crowded. Products overlap, lean, sit behind each other and get turned sideways by shoppers. Dense scenes with 100 or more objects per image are a separate detection problem from the clean, sparse scenes most models are trained on.
Every store looks different. Lighting color, shelf height, camera angle, glare off freezer doors and price label formats all vary between locations. This is domain shift, and it's the main reason pilots don't scale. The public GroZi-120 dataset shows it in miniature: it pairs clean studio photos of 120 grocery products with real shelf photos of the same items, and models trained on the first set struggle on the second.
Every use case above depends on labeled images of real products in real conditions. There are three ways to get them, and most production teams end up combining all three.
Public datasets are the fastest way to start. SKU-110K contains 11,762 shelf images with about 1.7 million bounding boxes, roughly 147 products per image, which makes it the standard benchmark for dense shelf detection. RPC (Retail Product Checkout) has 83,739 images of 200 SKUs for checkout recognition. GroZi-120 covers the studio-to-shelf gap described above.
The limits show up quickly. These datasets contain someone else's products, label formats and store layouts. They're useful for pretraining a detector, but a model that has to recognize your exact SKUs still needs your own data.
The usual next step is to photograph shelves and pay annotators to draw boxes. For retail this gets expensive fast. A single dense shelf image can hold over a hundred products, each needing a box and the correct SKU label, and telling two near-identical variants apart requires annotators who know the catalog. Then packaging changes and part of the work has to be redone.
Rare cases are the bigger problem. A product lying on its side, a shelf half-blocked by a shopping cart, a new seasonal pack in its first week on shelf: these show up rarely in collected photos, yet they're exactly where models fail.
Synthetic data replaces photography with rendering. You build 3D models of products, place them on virtual shelves, vary lighting, camera angle, facing count and occlusion, and render thousands of images. Every image comes with perfect labels automatically: bounding boxes, segmentation masks, SKU IDs and depth, with no annotator involved.
Retail has an advantage here that most industries don't. Many brands and retailers already have 3D models of their products, made for product pages, configurators and AR. The same assets can feed a training pipeline. When packaging changes, you update the texture on the model and regenerate the dataset in hours instead of re-shooting and re-labeling shelves.
At Vivid 3D we generate shelf scenes like this: multiple SKUs per shelf, controlled occlusion, varied store lighting and camera positions, exported in COCO, YOLO or KITTI format. The approach behind it is covered in more depth in our guides on synthetic data for object detection and segmentation and combining synthetic and real data.
Synthetic data works best as a complement. The common pattern is to pretrain on a large synthetic set that covers every SKU and edge case, then fine-tune on a smaller set of real store photos to close the remaining domain gap.
Cameras in stores raise a fair question from shoppers: am I being identified? Most of the use cases in this article don't need to identify anyone. Shelf cameras point at products, and traffic analytics track anonymous body positions.
Facial recognition is a different matter, and regulators have acted on it. In December 2023 the US Federal Trade Commission banned Rite Aid from using facial recognition for five years. The FTC found that the system had falsely flagged shoppers as suspected shoplifters, and disproportionately so in some communities. In the EU and UK, biometric identification falls under GDPR rules on special category data.
The practical rules are simple. Point cameras at shelves rather than faces where possible, process video at the edge, keep only the structured output, and blur or skip faces in any stored footage. Training on synthetic data helps here too: no real shopper ever appears in the dataset.
Step four is where most retail projects stall. If your team is building shelf monitoring, product recognition or checkout models and the data side is the bottleneck, Vivid 3D's synthetic data platform turns your 3D product models into labeled training sets for computer vision in retail, and regenerates them when your catalog changes.
Not fully. Computer vision mostly changes what cashiers do rather than removing them. Amazon pulled its Just Walk Out system from US Amazon Fresh grocery stores in 2024, and most self-checkout areas still need attendants to handle flagged transactions, age checks and problems. Fully cashierless stores work best in small formats like stadium and airport shops.
Rite Aid is the best-known US case. It used facial recognition in hundreds of stores until the FTC banned it from the practice for five years in 2023 over false matches. In the UK some retailers use Facewatch, which matches faces against a watchlist of known offenders. Most large chains avoid facial recognition and use product-focused cameras instead.
Production systems detect shelf gaps and product facings reliably, but SKU-level accuracy depends on how many near-identical products the model must separate and how well its training data matches real store conditions. Accuracy usually drops when a model moves to new stores with different lighting and layouts, which is why testing across varied locations matters.
There is no fixed number, but a practical guide is hundreds of labeled examples per SKU across different angles, lighting and occlusion levels. With thousands of SKUs that quickly reaches millions of labels. Teams cut the cost by pretraining on public or synthetic data and fine-tuning on a smaller set of real store images.
