Retail Robotics Solution
AI-powered checkout platform for restaurants
Why restaurants needed automatic food recognition
In traditional self-service systems, the user has to manually search for dishes, scan barcodes, or interact with staff.
This leads to:
- increased service time;
- queues during peak hours;
- extra workload on employees;
- errors in identifying dishes.
Project Goal:
Create a computer vision system capable of automatically identifying dishes on a tray and shortening the user journey from food selection to payment.
Retail Robotics Solution is an AI-powered platform that automates the checkout process in canteens and self-service restaurants. The system uses computer vision and neural networks to instantly recognize dishes on a tray — no barcodes, no manual input.
The main challenge: designing an interface that gracefully handles AI recognition edge cases (model uncertainty, dish overlap, age restrictions) while remaining intuitive for non-technical users.
My Responsibilities
UX/UI design of the self-service terminal;
designing user payment flows;
designing food recognition screens;
analyzing edge cases;
designing user interaction with AI;
working with business requirements;
detailing interfaces for real restaurant scenarios;
participating in the development of the ad module.
Self-service checkouts today are slow
In canteens, thousands of people stand in queues every day. Cashiers manually identify dishes, make pricing errors, and confuse portions.
Constraints and Requirements
The design process took into account several key constraints:
the user should not require complex training;
the interface must be intuitive from the first interaction;
the system must gracefully handle recognition errors;
the payment flow must remain fast even in edge cases;
the interaction must be equally clear to users of all ages and experiences.
Why Food Recognition is Hard
Unlike barcodes, real food is visual chaos. The neural network must handle dozens of variables.
Visual Similarity
Some dishes look almost identical. Pureed soups and porridges, different types of pastries. AI must understand the finest distinctions.
Object Overlap
Plates on trays often stack, bread covers side dishes, cutlery overlaps food items.
Portion Variations
The same dish looks different depending on the cook, portion size, lighting, and plate.
Scenarios the system faces
All of this shows that the problem is much more complex than simple item scanning.
How It Works
Place Tray
Place your food tray under the camera and press the button.
Review Order
Neural network instantly identifies dishes. Check that everything is correct.
Pay
Choose a payment method and tap your card on the terminal.
Done
Take your receipt and enjoy your meal!
Speed
The whole process takes 10–15 seconds
For comparison: a regular human cashier takes 45 to 60 seconds per guest. We sped up the checkout process by 4-5x.
User Journey
Edge Cases — The Real Challenge
The main challenge of the project was not the ideal scenario, but handling real situations that regularly occur in self-service restaurants.
The main flow is simple and covers most situations. But the real designer work is in edge cases. What happens when AI is uncertain? When the tray is empty? When there's alcohol on the tray?
Empty Tray
The AI camera detected no dishes. Instead of a boring system error — a friendly state with clear actions: retry scanning or cancel the order.
Dish Clarification
The neural network recognizes 98-99% of dishes perfectly, but for the rare remaining cases, we have a fallback scenario. If AI recognizes the category (e.g. soup) but is unsure about the specific dish, we show the camera photo on the left and possible menu options on the right. User chooses in 2 seconds.
Object Overlap
Plates are too close or stacked on each other, so AI can't correctly recognize all objects. In this scenario, the system politely asks the user to spread out the dishes and retries the scan.
Additional value of the payment screen
Besides the main payment function, the terminal allows using the screen as an advertising platform.
Restaurants can place partner advertising content and use the waiting time for additional communication with visitors.
Results
As a result, the project combined computer vision technologies and a convenient user interface, allowing to create a fast and intuitive self-service flow.
Recognition time for all dishes
Dish recognition accuracy
Faster than regular checkout
Monthly terminal cost
Successful Launch in Europe and the US
Today, the Retail Robotics Solution system is successfully implemented and operating in canteens and self-service restaurants across Europe and the US. The product has proven its efficiency in international markets, processing thousands of orders daily and reducing service time.
Other Screens
Enjoyed this case study?
I am open to new challenges and ready to discuss your next project.
Contact me on Telegram
