The Transreality Interaction Platform: Enabling Interaction across Physical and Virtual Reality

The convergence of the Internet of Things (IoT) and interactive systems will enable future interactive environments which transcend physical and virtual reality. Embedded Things provide sensors and actuators to virtualize the physical environment, while Interactive Things extend the virtualized environment with modalities for human interaction, ranging from tangible and wearable interfaces to immersive virtual and augmented reality interfaces. We introduce the Transreality Interaction Platform (TrIP) to enable service ecosystems which situate virtual objects alongside virtualized physical objects and allow for novel ad-hoc interactions between humans, virtual, and physical objects in a transreality environment. TrIP provides a generalized middleware platform addressing the unique challenges that arise in complex transreality systems which have yet to be fully explored in current IoT or HCI research. We describe the system architecture, data model, and query language for the platform and present a proof-of-concept implementation. We evaluate the performance of the implementation and demonstrate its use integrating embedded and interactive things for seamless interaction across physical and virtual realities.