A Scene Understanding and Positioning System from RGB Images for Tele-meeting Application in Augmented Reality

As augmented reality (AR) technology advances, there is a growing demand to apply it to various applications. With the outbreak of the COVID-19 epidemic in 2020, online meetings have become increasingly common to prevent human contact, creating an opportunity to implement AR technology and its related applications. In this paper, we propose a new AR solution for tele-meeting applications that combines neural networks and simultaneous localization and mapping (SLAM) to achieve scene understanding and user localization using only RGB images. Our system offers the flexibility to develop the target application solely based on our custom devices/software, rather than relying on existing AR software development kits (SDKs) with their limitations. Existing SDKs, such as ARCore, can only be used on officially certified devices by Google, and developing custom AR kits to resolve compatibility issues among multiple technologies and devices can be challenging and time-consuming. This work presents a new system to address the challenges of scene understanding and user positioning for realizing tele-meeting applications, which can also be used as an AR development kit on any device with a camera. We conducted several experiments on the system modules to verify its performance, and the results show that our system provides an efficient and stable user experience, making it a promising technology and application.