Dual Face Interaction in Handheld Augmented Reality Environments

Sophisticated multimodal interfaces are the key to success in future computer development and many ideas have been proposed so far. Augmented reality (AR) is such an idea. Computer-generated digital objects are mixed with objects in the real world, and they are perceptually assimilated. People don't need to be aware of the computer. What to do for them is just behave as usual in the real world. However, interaction with the objects is limited in existing AR systems. In fact, many of the systems allow the user just to change his/her viewpoint to real world objects. There are some other systems which accept manipulation of virtual objects, but they assume users wear or hold special devices. In this paper we propose a handheld AR interface supporting two interaction techniques based on the understanding that both real and virtual objects are compatible in manipulation. One of the techniques is for the user to directly manipulate virtual objects by his/her finger. One more technique concerns indirect manipulation of virtual objects. Motion of a handheld device serves as a trigger to make a certain change to target virtual objects.