Inter-Pose: An Interactive 3D Modeling Based on 6D Pose Estimation

3D modeling serves as a cornerstone in various domains such as XR, digital content, and interactive media, enabling users to simulate and perceive immersive 3D world. Traditional 3D modeling using CAD software often proves labor-intensive and detached from real-world scenarios. In response, we introduce Inter-Pose in this work, a novel interactive 3D modeling approach leveraging 6D pose estimation. It enables users to design 3D scenes interactively, akin to games like Minecraft, by predicting layouts based on 2D projections of 3D bounding box vertices. Users can arrange models in the real world, and corresponding scenes are presented on computers, overcoming challenges posed by conventional 3D modeling methods. We evaluate the end-to-end trainable 6D pose estimation module in Inter-Pose on the InterCube dataset, demonstrating its rapid and accurate performance. Operating at 63 fps on a GTX 1050ti GPU, Inter-Pose proves suitable for real-time applications. Our contributions include the development of an interactive modeling approach, a precise 6D pose estimation module, and the creation of the InterCube dataset.