VR Visualization of Heritage Structures at Different Scales via Mode Switching

Advances in 3-dimensional (3D) visualization technologies, especially, in virtual reality (VR) headsets, have enabled users to remotely, yet immersively, experience heritage structures. This has obviated the need for physical travel, especially to remote or inaccessible locations, and for relocating historical artifacts to museums. While lifelike visualization does depend on a display such as a VR headset, one would primarily need a comprehensive yet high-resolution 3D model in the first place. However, generating highly detailed 3D models of large-scale heritage structures poses significant computational and time challenges. Conversely, practical 3D models generated from multiple views using available resources tend to lack finer details. Noting that the original images used for creating the 3D model usually contain such details, we propose to switch from rendering the 3D model to a suitable 2D image when the user requires detailed/closeup information about intricate patterns and motifs. To achieve this, we employ learning-based feature matching and pose alignment between the rendered 2D views and the real-life 2D images. Further, a VR add-on enhances the immersive experience. Our system is readily accessible online via GitHub.

I. Introduction

A visit to heritage sites exposes the casual visitors to the architecture and other artifacts of the past, and excites them by creating connections to historical events, accounts, and even anecdotes. Students and professionals of related disciplines, such as archaeology and architecture, benefit by making deeper studies and possibly finding connections among various historical trends and even modern ones. However, physical visits, especially to remote sites, can be challenging and even unaffordable. Further, removing artifacts from their original location may not always be feasible/desirable. In response, major museums, including the Louvre at Paris, have begun virtual exhibition [1]. The Open Heritage Program, a collaboration between Google Arts & Culture and CyArk, has introduced virtual tours of numerous UNESCO world heritage sites [2]. In India, the Digital Hampi initiative of the Department of Science and Technology, Government of India, digitally modeled and displayed the eponymous heritage site, including at the International Heritage Symposium and Exhibition (IHSE), 2020 [3]. Virtual exhibitions, in addition to addressing the aforementioned issues, have also opened up new possibilities. Specifically, large monuments can now be experienced at various scales, from virtual walkthroughs around the entire structure as well as larger parts such as rooms, domes and minarets, to views and geometry of ornate columns, to even the minutest details of various patterns and motifs, all on the same visualization platform such as a virtual reality (VR) headset (see, for example, Figure 1). This remarkable functionality, however, throws up certain new challenges, tackling one of which constitutes the main objective of this paper.