Simultaneous tele-visualization of construction machine and environment using body mounted cameras

During the teleoperation of construction machines, it is highly needed to visualize the machine body itself. This is because the operator needs to perform a visual confirmation of the construction machine. In addition, information of crawler parts' shoe slip is indispensable. Therefore, not only the surrounding environment but also the vehicle body are needed to be visualized. This paper proposes a tele-visualization system for both the construction machine and the surrounding environment from an arbitrary viewpoint by using mounted cameras. In the proposed method, both the construction machine and the surrounding environment are expressed as three-dimensional mesh models. In order to capture the actual scenes of the construction machine and the surrounding environment, the positions and orientations of the mounted cameras are adjusted. Simultaneous tele-visualization is achieved by mapping the image data captured from the scenes to the correct positions on the mesh models, after estimated the correspondence between the camera images and the mesh models.