Realtime Object Recognition Method Inspired by Multimodal Information Processing in the Brain for Distributed Digital Twin Systems

Recently, digital twins have been paid much attention as a major application towards Beyond 5G/6G network, and real-time object recognition methods are key technology to digitize the real world as a digital twin. However, it is challenging to make a fast and accurate decision on what the object is from real-time streaming information such as video because accurate object recognition algorithms require a huge computation. To satisfy delay requirement of digital twin applications, such computations have to be moved from cloud to edges or even small terminal devices, where computing capacity is very limited. Thus, recognition mechanisms have to be simplified for small devices but they would result in degraded accuracy. In this paper, we focus on the multimodal information processing mechanism of the brain, which makes decisions based on multiple types of uncertain observed information, to improve accuracy of simplified recognition mechanisms. We first propose a unimodal object recognition mechanism based on the Bayesian attractor model, which continuously recognizes objects from noisy streaming media data. Then, we extend the mechanism with Bayesian causal inference to fuse the results of unimodal media recognition. Through computer simulations, we show that our proposed method identifies an object accurately and quickly from uncertain observed information.