Enhancing First-Person View Task Instruction Videos with Augmented Reality Cues

This research investigates enhancing first-person view (FPV) task instruction videos by applying Augmented Reality (AR) visualisation of spatial cues. With personal mobile devices, recording and sharing a video clip has become very easy, and how-to videos are becoming popular on social video sharing services. Instructional videos are actively used not only in formal education and training, but also in everyday life. However, video clips are limited to two-dimensional representation of the task space, making it hard for the viewer to follow and match the objects in the video to those in the real world task space. We propose augmenting task instruction videos with AR visualisation of spatial cues to overcome this problem, focusing on creating and viewing FPV instruction videos. We designed and implemented a prototype system, AR Tips, which allows users to capture and share augmented FPV instruction videos on a wearable AR device. We conducted a user study to evaluate the benefit of our approach, and the results showed that with the help of augmented spatial cues users better understood the instructions, performed the tasks faster with fewer errors, and had lower mental effort.