UVM++: A Large-Scale Benchmark for Beverage Recognition in Intelligent Vending Machine

In recent years, artificial intelligence (AI)-driven computer technology has become popular in many practical applications in the retail industry. In particular, the development of face recognition, mobile payment, cloud computing and other technologies has given birth to three new forms of consumption related to “unmanned retail”, i.e., unmanned convenience stores, unmanned vending machines and unmanned vending racks. Due to the sudden outbreak of COVID-19, the characteristics of unmanned retail such as intelligent operation, convenience and a lack of contact have allowed the public to gradually become used to this approach. An unmanned vending machine (UVM) can be reconfigured from a refrigerated cabinet for selling commodities such as beverages. It can take photos of the goods purchased by customers, and can recognize these commodities based on information from the collected images. However, some commercial UVMs are subject to strong restrictions on the types and placement of goods, making them difficult to achieve data collection and convenient operation. In this research, we therefore design a system for automated data collection and annotation. The mechanical structure of our system is designed to mimic the placement of drinks and the scene of each drink with other different types of drinks around it. We construct a large-scale benchmark dataset called UVM++, which is composed of several sub-datasets collected from different scenes. Extensive experimental results are presented and the performance of popular object detection methods on this task is examined. In particular, we perform an empirical study of cross-scene detection by various object detectors. Our findings suggest that the proposed data annotation system and the constructed benchmark dataset, UVM++, can be useful for performing various tasks in the environment of a UVM, thus driving development in the field of new retail.

I. Introduction

With the unprecedented development of artificial intelligence (AI) and communication technology, a revolution has taken place in the traditional retail industry. Initially, Zakkour [1] has reported the concept of “new retail”, consisting of a road map of the modern retail industry which connects online and offline life, combines commodities and services, and provides digital management and intelligent operation. Unmanned retail machines [2], hitherto regarded as a novelty in the development of new retail services, are used to explore this new “mode” of intelligence and self-service.