Environment Understanding Framework for Intelligent Mobile Systems

Environment perception is a key requirement for intelligent mobile system. Vision based methods that support this task have advanced impressively in the past years, mainly due to the advent of deep learning techniques and dedicated hardware. Still, the state of the art techniques for simultaneously solving the problem of object detection as well as that of semantic segmentation are computationally expensive and, thus, not straightforward to use by intelligent mobile systems. In this paper, we introduce a novel framework for environment understanding based on semantic instance segmentation and pre- and post-processing techniques to speedup the instance detection and classification, while keeping the accuracy at state of the art levels. The designed method employs an initial fast segmentation. This procedure splits the image into several regions based on U-disparity maps. Afterwards, each region is fed through a semantic instance segmentation network. Further, to avoid multiple predictions due to segmentation and/or prediction, a filtering heuristic is used to post-process the resulting instances. Finally, we evaluate the proposed framework using available benchmarks, highlighting the real-time operation capability and the solution’s accuracy.