A Simple Method on Generating Synthetic Data for Training Real-time Object Detection Networks

Environment recognition has been an important topic ever since the emergence of augmented reality (AR). For better experience in AR applications, environment recognition should be provided fast in real-time, where real-time object detection technologies could fulfill this requirement. However, training object detectors for AR specific scenarios are often troublesome. The real-time nature of AR produces visual degradations such as motion blur or occlusion by interaction, which make detectors trained with plain data difficult to detect objects exposed in such complex situations. Also, since gathering and labeling training data from scratch is a heavy burden, we need to resort to synthesized training data but previous synthetic data generation frameworks do not consider the aforementioned issue. Therefore, in this paper, we propose a new synthetic data generation framework which includes visual variations such as motion blur and occlusion occurred by distractors. By this simple modification, we show that including such variated data to the training dataset could dramatically improve realtime performance of object detectors by a high margin. Also, we stress that synthesizing training data with no more than three objects per image can achieve competitive performance compared to detectors trained with over four present in a single image. Experimental results both quantitatively and qualitatively supports our statements and shows the superiority of our method.

I. Introduction

Augmented reality (AR) has received significant attention across the industry and academic society [1]–[4]. Recently, AR technologies pursue methods for more realistic experience, which has also been studied across various domains [5]–[11]. To meet the demand for more realistic AR applications, developers started to pursue higher level user interactions. For realistic experience in such interactions, recognizing the surrounding environment in real-time is highly necessary. A suitable technology to satisfy this requirement is object detection. Object detection can be considered the most basic form of environment recognition since knowing which object is present in the current scene can provide plenty information about the environment. Also, object detection have shown astonishing development [12]–[19] and now one stream of the detection systems guarantee real-time (>30 fps) performance with high reliability [17]–[19]. By customizing these object detection systems, we can provide better experience reliably in real-time for AR applications.