Development of VR visualization system including deep learning architecture for improving teleoperability

The recognition and visualization of the situations in remote areas help the tele-operator work more effective and safe. This paper proposes the visualization system which can acquire and display 2D and 3D image-based environmental information for the operator. The proposed visualization system consists of egocentric viewer and exocentric viewer system. The egocentric viewer system shows VR information based on the object detection results using deep learning architecture. The exocentric viewer system shows the pose and space constraint information of the remote operation robot to be controlled on the 3D point cloud data scene that is capable of point-of view conversion.