Simbe Wins Patent for Tracking Freshness of Fruits and Vegetables
Simbe Robotics recently received a U.S. patent for using an autonomous mobile robot and computer vision system to track the freshness and ripeness of fruits and vegetables on grocery store displays.
According to the data of the Economic Research Service of the U.S. Department of Agriculture (ERS), in 2010 a total of 133 billion pounds (31%) of the 430 billion pounds of food available at retail and consumer levels went unused, with an estimated retail value of $162 billion. Supermarkets' average loss rate was estimated at 11.6% for 31 types of fresh vegetables.
Grocery stores operate on thin margins, so every small saving matters. This is a potentially huge market opportunity where Simbe Robotics is already present. Simbe recently announced the expansion of its inventory robots to 100% of Schnucks grocery stores in the US.
“We are facing a "new normal" in the grocery industry, and Tally has played an important role in helping us continue to deliver an exceptional in-store experience while addressing new operational challenges. Having deployed Tally across all our stores, we have fully integrated this information into our supply chain and expanded our ability to use real-time data for decisions that affect revenue. Tally has become an integral part of our stores, optimizing operations and ultimately creating a better in-store experience for our customers and team members”
The difficult part of this application is computer vision and creating an algorithm that can determine when food begins to spoil, and then when to send an alert to a fresh produce associate to fix the situation. In the patent application, the company describes using hyperspectral imaging to determine the freshness of fruits and vegetables and characterize their quality.
The Simbe Tally robot is one of the market leaders in grocery store inventory management. Until now, Tally could track inventory of all shelf-stable foods and everyday goods in the store. But inventorying fresh produce and its tendency to spoil has long been a challenge, so this was not a capability of the Tally robot. Until now.
“Schnucks is a prime example of thoughtful adoption of retail technology, and it is an honor for us to be their partner on this journey toward a better store experience through access to richer data. This expansion is an important milestone for Simbe, Schnucks and the retail technology industry as a whole. It demonstrates that carefully designed robots are a critically important tool for retailers, helping them improve net profit, support their teams and preserve an exceptional shopping experience both in stores and online”
With the patent in hand, Simbe can begin the process of commercializing this idea. It is quite likely the company is already using it in laboratory conditions, but the real proof will be testing images and analysis in real grocery store conditions with lighting that changes throughout the day.
The Robot Report asked the company for comment, but Simbe is currently keeping quiet about its commercialization plans.
This application has been the "holy grail" since the early days of in-store inventory management with mobile robots. The biggest obstacle was solving the problem of lighting, imaging and analyzing fresh food on displays. Several competitors in this area have chosen to place products on standard shelves. However, all these competitors noted that capturing images of fresh fruits and vegetables remains a difficult task.
First, displaying fruits and vegetables on a display requires a different lens and depth-of-field settings than displaying them on a shelf. This requires the camera to look down from a different viewpoint than the camera looking at items on shelves.
This is because fruits and vegetables are usually arranged at different angles when products are stacked and presented to shoppers. Other challenges include reflections from overhead lighting used to illuminate the merchandise, as well as inconsistent reflections from moisture on the surface of items due to frequent water misting.
The long and complex patent runs 32 pages. It covers the design of an application that assesses the condition of fruit, identifies spoiling fruit and subsequently alerts a worker to check and/or replace spoiled fruit. The result of the new features will be the ability to warn a fresh produce associate that displayed items need to be removed from sale and replaced with fresher offerings.
Simbe's ultimate goal is to make more effective use of its mobile robot in the store, together with the ability to cover more SKUs (stock keeping units) in the store, but especially to support one of the most labor-intensive processes in the store: managing fresh produce displays.
