Figure 3: Decision Framework for designing a smart integrated shelf monitoring solution
- 4Ws – What, Where, Why, Who
- What: What tasks need to be monitored at the shelf?
E.g. – Empty shelf replenishment, restoring location of misplaced items, BOPIS orders (Buy Online Pickup in Store) waiting to be picked up, and items with outdated prices
- Where: Where are we monitoring – which store, aisle sections and categories?
E.g. – It is best to start a pilot with your busiest aisles stocking fast moving items such as cereals, beverages, ready to eat, etc.
- Why: Why are we monitoring these tasks - what are the benefits?
E.g. - There are high chances your associates are making multiple trips to the shelf per day to fix a planogram. In a single trip, detecting non-planogram tasks that are ageing, such as BOPIS orders awaiting picking, can improve operational throughput
- Who: Who will monitor – associate, floor supervisor, deputy or store manager?
E.g. – A floor supervisor may just need to monitor shelves in periodic hours and notify associates, whereas an associate will need to execute the task himself
- How? User journey & business process
How should the business user monitor the shelf? Should we remotely monitor from a back room or should we focus on mobility, and conduct patrol rounds? For an effective solution we need to look at both the user journey and the business process. Associates or floor supervisors can be equipped with smart mobile apps for detecting shelf compliance issues, and triggering notifications or alerts to an integrated task management system.
- Choice of technology to accurately handle the complexities: Choice of technology is very crucial because of the complexities involved in accurately detecting products and their shelf locations from images. The complexities that pose challenges in object detection from shelf images include –
- Varied size and shapes of products
- Multiple facings of a SKU
- Varying light intensities on the packaging surface
- Packaging variations for the same SKU
- SKUs appearing at the corner of an image
- Image distortions impacting quality
Computer vision combined with deep reinforcement learning has proven very effective in successful detection of SKUs and their shelf locations from images amidst these complexities. Artificial intelligence (AI) has elevated detection accuracy to almost 100% in certain cases (E.g. Symphony Retail), compared to 80% accuracy observed with image processing techniques. With computer vision, it’s possible to detect the objects and their locations from a shelf image with a reasonable amount of precision. When these computer vision models are further enriched by data annotation, deep learning and OCR (Optical Character Recognition), the precision and confidence of object detection amplify tremendously. In addition, reinforcement learning can improve accuracy even further by giving associates a chance to provide a feedback if the object was correctly detected or not, and then the AI self-learns from the feedback.
With compliance done right with AI, retailers can consider stepping up associate experience even further by incorporating augmented reality that can overlay virtual instructions for associates on the physical shelf when they turn on the camera of the mobile device. We demonstrated an AR based associate experience at NRF2020 for guided shelf monitoring, as cited in Figure 2.
- User adoption policy: Driving user adoption is a daunting challenge for large retailers with footprints of over 1,000 stores and more than 50,000 associates. It’s a matter of proactively grabbing a tablet, doing a floor round to detect shelf anomalies. Gamification and AR can play a big role in incentivizing associates. Imagine an associate getting rewards for detecting shelf anomalies such as misplaced items. AR based wayfinding for reaching the right shelf brings a lot of fun to mundane store operations.
Role of retail segment and business strategy
The industry segment and business strategy play a great role in how this solution can be customized for different retailers. Imagine a consumer electronics retailer who has invested large space in experience zones within their stores. Such stores usually have a dedicated backroom holding inventory for online orders. Hence, an associate would only require alerts of misplaced items or empty shelves on the experience zone. Following the “4W” element of the framework, online order picking would not be a relevant task that needs to be integrated to the solution.
Let’s take a different example – a grocery retailer for whom convenience and operational efficiency are critical. The same shelf fixtures would have products on display as well as products allocated to online orders. Hence, online order picking is a relevant task that needs to be monitored so that the right associates can be notified through an alert.
To get started with a successful shelf monitoring program, retailers need to look at the final and most important piece of the puzzle – a partnership. The success of this program depends on best practices from multiple fields like visual merchandising, artificial intelligence, process design, complex system integrations, and associate empowerment. The right culture can be cultivated by having a cross-functional talent pool comprising of store managers, business leadership, product managers, retail functional consultants, AI engineers, and system architects. After all, great successful hi-tech business solutions require tremendous depth of industry experience, functional and technical expertise. Only partnerships can bring such expertise under one roof.