Deliverable | Retail technology | KPI framework | Reference design, no targets or results
How to use this catalogue
The author contributed domain expertise to solutions of this kind at a medium-scale, product-based company based out of Dallas and Coimbatore. This catalogue is a reference design. It lists measures that a retailer or vendor can choose from when running a vision-based checkout or an unattended store. It sets no targets and reports no results, because targets depend on a store’s own baseline.
Pick a small set for a pilot: one or two from each family below, plus the gates the programme agrees in advance. The recognition and exception families explain whether the system works. The dispute and loss families explain whether it is safe to scale. The data operations family explains whether it will keep improving. Each measure should be reported against a baseline, split by site and item class, and reviewed weekly during a pilot.
Every measure includes a definition, a unit, a suggested review frequency and a typical owner. Frequencies and owners are suggestions.
Recognition accuracy
| ID | Measure | Definition | Unit | Review | Owner |
|---|
| K01 | Item-level precision | Share of recognised items that are correct | % | Daily | Model owner |
| K02 | Item-level recall | Share of items on the tray that were recognised | % | Daily | Model owner |
| K03 | Basket-exact rate | Share of baskets where every item and quantity is correct | % | Daily | Model owner |
| K04 | Top-1 class accuracy | Share of items where the top prediction is correct | % | Per release | Model owner |
| K05 | Top-3 class accuracy | Share of items where the right class is in the top three | % | Per release | Model owner |
| K06 | Fine-grained confusion rate | Share of errors between look-alike items (sizes, flavours, variants) | % | Weekly | Model owner |
| K07 | Occlusion error rate | Error rate on baskets with touching or stacked items | % | Weekly | Model owner |
| K08 | Unrecognised-item rate | Share of items the model declines to name | % | Daily | Model owner |
| K09 | Unknown-item false match rate | Share of out-of-catalogue items named as a catalogue item | % | Weekly | Model owner |
| K10 | Loose and unpackaged item accuracy | Accuracy on produce, bakery and other barcode-free goods | % | Weekly | Model owner |
| K11 | Quantity count accuracy | Share of multi-unit lines with the correct count | % | Daily | Model owner |
| K12 | Calibration error | Gap between stated confidence and observed accuracy | score | Per release | Model owner |
Basket and checkout quality
| ID | Measure | Definition | Unit | Review | Owner |
|---|
| K13 | Basket acceptance rate | Share of baskets accepted without human help | % | Daily | Store ops |
| K14 | Signal agreement rate | Share of baskets where vision, weight and price rules agree | % | Daily | Store ops |
| K15 | Weight mismatch rate | Share of baskets where weight disagrees with recognition beyond tolerance | % | Daily | Store ops |
| K16 | Price rule violation rate | Share of baskets that fail a price or promotion rule check | % | Daily | Pricing |
| K17 | Promotion application accuracy | Share of eligible baskets with the correct promotion applied | % | Weekly | Pricing |
| K18 | Tray-to-receipt time | Median time from placing items to receipt | seconds | Daily | Product |
| K19 | Re-present request rate | Share of baskets where the shopper is asked to re-present an item | % | Daily | Product |
| K20 | Abandonment rate at checkout | Share of started checkouts not completed | % | Daily | Product |
| K21 | Payment failure rate | Share of accepted baskets whose payment fails | % | Daily | Payments |
| K22 | Receipt accuracy complaints | Complaints about receipt content per 1,000 baskets | count | Weekly | Customer care |
| K23 | Age-restricted item check pass rate | Share of restricted items sold with a completed check | % | Daily | Compliance |
| K24 | Bag and container handling accuracy | Accuracy when items arrive in bags or containers | % | Weekly | Model owner |
Exceptions and assistance
| ID | Measure | Definition | Unit | Review | Owner |
|---|
| K25 | Exception rate | Baskets sent to a person per 100 baskets | rate | Daily | Store ops |
| K26 | Exception rate by cause | Exceptions split by recognition, weight, price, payment and other | % | Weekly | Store ops |
| K27 | Median time to resolve | Median time from exception raised to resolved | seconds | Daily | Store ops |
| K28 | 95th percentile wait | Wait for the slowest 5 percent of exceptions | seconds | Daily | Store ops |
| K29 | Attendant touches per 100 baskets | Staff interventions per 100 baskets | rate | Daily | Store ops |
| K30 | First-contact resolution rate | Exceptions resolved without escalation | % | Weekly | Store ops |
| K31 | Remote assistance share | Share of exceptions resolved remotely | % | Weekly | Store ops |
| K32 | Exception reopen rate | Exceptions reopened after resolution | % | Weekly | Store ops |
| K33 | Shopper-initiated help requests | Help requests per 100 baskets | rate | Weekly | Product |
| K34 | Staff override rate | Share of exceptions where staff override the system suggestion | % | Weekly | Model owner |
| K35 | Exceptions converted to training examples | Share of resolved exceptions stored as labelled data | % | Weekly | Data ops |
| K36 | Exception queue backlog | Open exceptions at peak | count | Daily | Store ops |
Disputes and customer trust
| ID | Measure | Definition | Unit | Review | Owner |
|---|
| K37 | Dispute rate | Disputed baskets per 1,000 baskets | rate | Weekly | Customer care |
| K38 | Dispute upheld rate | Share of disputes resolved in the shopper’s favour | % | Weekly | Customer care |
| K39 | Time to dispute resolution | Median time from dispute to refund or closure | hours | Weekly | Customer care |
| K40 | Wrongful-charge rate | Baskets charged for items not taken per 1,000 | rate | Monthly | Customer care |
| K41 | Missed-charge rate | Baskets that omitted items taken per 1,000 (from audits) | rate | Monthly | Loss prevention |
| K42 | Repeat-use rate | Share of shoppers who use the lane or store again within 30 days | % | Monthly | Product |
| K43 | Satisfaction score | Post-checkout satisfaction rating | score | Weekly | Product |
| K44 | Trust complaints | Complaints about accusations or monitoring per 1,000 baskets | count | Monthly | Customer care |
Loss and shrink
| ID | Measure | Definition | Unit | Review | Owner |
|---|
| K45 | Total shrink rate | Inventory loss as a share of sales | % | Monthly | Loss prevention |
| K46 | Lane-attributed unknown loss | Unknown loss attributed to the lane or format | % of sales | Monthly | Loss prevention |
| K47 | Non-scan or non-recognition rate | Items taken but not registered (audit-based) | % | Monthly | Loss prevention |
| K48 | Item substitution rate | Cheaper item presented for a dearer one (audit-based) | rate | Monthly | Loss prevention |
| K49 | Audit mismatch rate | Mismatch found in sampled basket audits | % | Monthly | Loss prevention |
| K50 | Malicious loss share | Share of lane losses judged intentional | % | Quarterly | Loss prevention |
| K51 | Accidental loss share | Share of lane losses from honest error | % | Quarterly | Loss prevention |
| K52 | High-risk item loss rate | Loss rate on the top 50 items by theft risk | % | Monthly | Loss prevention |
| K53 | Door or exit breach events | Exits without a completed basket in controlled formats | count | Weekly | Store ops |
| K54 | Wastage rate | Spoiled or damaged stock as a share of sales | % | Monthly | Store ops |
| K55 | Refund abuse indicators | Refund requests flagged by pattern rules | count | Monthly | Loss prevention |
| K56 | Loss by shift or hour band | Loss split by time of day | % of sales | Monthly | Loss prevention |
Inventory and restocking
| ID | Measure | Definition | Unit | Review | Owner |
|---|
| K57 | On-shelf availability | Share of catalogue items in stock at a sampled time | % | Daily | Store ops |
| K58 | Out-of-stock duration | Median time an item stays out of stock | hours | Weekly | Store ops |
| K59 | Inventory record accuracy | Share of items whose system count matches a physical count | % | Monthly | Store ops |
| K60 | Planogram compliance | Share of shelf positions holding the planned item | % | Weekly | Store ops |
| K61 | Restock task completion time | Median time from restock task raised to done | minutes | Weekly | Store ops |
| K62 | Restock error rate | Restock actions that changed shelf layout without update | % | Weekly | Store ops |
| K63 | Days of cover | Average days of stock on hand | days | Weekly | Supply chain |
| K64 | Stock variance after audit | System count minus counted stock, absolute | units | Monthly | Store ops |
Operations and availability
| ID | Measure | Definition | Unit | Review | Owner |
|---|
| K65 | System uptime | Share of opening hours the format is fully operating | % | Daily | Engineering |
| K66 | Edge-offline operation share | Time operating on local fail-safe due to network loss | % | Weekly | Engineering |
| K67 | Camera or sensor fault rate | Faults per device per month | rate | Monthly | Engineering |
| K68 | Mean time to repair | Median time from fault to restoration | hours | Monthly | Engineering |
| K69 | Door unlock latency | Median time from authentication to door open | seconds | Weekly | Engineering |
| K70 | Basket decision latency | Median time to assemble and score a basket | seconds | Weekly | Engineering |
| K71 | Software release failure rate | Releases that need rollback | % | Monthly | Engineering |
| K72 | Support ticket volume | Support tickets per site per week | count | Weekly | Support |
Commercial and labour
| ID | Measure | Definition | Unit | Review | Owner |
|---|
| K73 | Sales per site per day | Daily sales by site | currency | Daily | Finance |
| K74 | Transactions per hour | Baskets per open hour | count | Daily | Finance |
| K75 | Average basket value | Sales divided by baskets | currency | Weekly | Finance |
| K76 | Staffed hours per 1,000 baskets | Labour hours needed per 1,000 baskets | hours | Monthly | Finance |
| K77 | Labour hours redeployed | Hours moved to other tasks | hours | Monthly | Store ops |
| K78 | Cost per basket | All-in operating cost per basket | currency | Monthly | Finance |
| K79 | Pilot-versus-baseline sales change | Sales change against the site’s own baseline | % | Monthly | Finance |
| K80 | Payback period | Time for net savings to cover investment | months | Quarterly | Finance |
Model and data operations
| ID | Measure | Definition | Unit | Review | Owner |
|---|
| K81 | Time to add a new item | Time from item first seen to live recognition | days | Weekly | Data ops |
| K82 | Catalogue coverage | Share of sold items the model can recognise | % | Weekly | Data ops |
| K83 | Images per item | Median labelled real images per catalogue item | count | Monthly | Data ops |
| K84 | Label-error rate | Share of audited labels found wrong | % | Per release | Data ops |
| K85 | Reviewer acceptance rate | Share of model proposals accepted unchanged | % | Weekly | Data ops |
| K86 | Review time per image | Median time to review one image | seconds | Weekly | Data ops |
| K87 | Holdout accuracy trend | Accuracy on the frozen real holdout across versions | % | Per release | Model owner |
| K88 | Synthetic-to-real gap | Accuracy on synthetic validation minus real holdout | points | Per release | Model owner |
| K89 | Drift alert count | Alerts for input or prediction drift | count | Weekly | Model owner |
| K90 | Retraining cycle time | Time from data cut to production model | days | Per release | Model owner |
| K91 | Packaging-change detection time | Time to detect a packaging change after it reaches shelves | days | Monthly | Data ops |
| K92 | Inter-reviewer agreement | Agreement between reviewers on a shared sample | % | Monthly | Data ops |
Privacy and governance
| ID | Measure | Definition | Unit | Review | Owner |
|---|
| K93 | Image retention compliance | Share of images deleted within the retention limit | % | Monthly | Privacy |
| K94 | Access audit findings | Unauthorised or unexplained access events | count | Monthly | Privacy |
| K95 | Signage compliance | Sites with required monitoring notices in place | % | Quarterly | Privacy |
| K96 | Data subject requests closed on time | Requests answered within the required period | % | Monthly | Privacy |
| K97 | Pilot gate status | Gates passed, at risk or failed | status | Weekly | Programme |
| K98 | Rollback readiness | Sites able to revert to conventional checkout within an agreed time | % | Quarterly | Programme |
Suggested pilot starter set
A pilot scorecard of about twelve measures is easier to run than the full list. A reasonable starter set draws from each family:
- Recognition: K03 basket-exact rate, K08 unrecognised-item rate.
- Basket quality: K13 basket acceptance rate, K18 tray-to-receipt time.
- Exceptions: K25 exception rate, K27 median time to resolve.
- Disputes: K37 dispute rate, K40 wrongful-charge rate.
- Loss: K45 total shrink rate, K49 audit mismatch rate.
- Data operations: K81 time to add a new item, K87 holdout accuracy trend.
Notes and method
This catalogue is a synthesis by the author from domain knowledge, informed by public research on self-checkout loss and retail checkout recognition: ECR Retail Loss, Global Study on Self checkout loss in Retail (https://ecrloss.com/research-paper/self-checkout-retail-loss-global-study/), the National Retail Federation National Retail Security Survey 2023 (https://nrf.com/research/national-retail-security-survey-2023) and the RPC retail product checkout dataset (https://arxiv.org/abs/1901.07249). The catalogue is a reference design. It contains no measured results, and no vendor claims were used.
Teams that want help turning this into a pilot scorecard can book a working session with the author through Topmate: https://topmate.io/raja_jeevan_kumar_maduri