A New People-Object Interaction Dataset and NVS Benchmarks

Recently, NVS in human-object interaction scenes has received increasing attention. Existing human-object interaction datasets mainly consist of static data with limited views, offering only RGB images or videos, mostly containing interactions between a single person and objects. Moreover, these datasets exhibit complexities in lighting environments, poor synchronization, and low resolution, hindering high-quality human-object interaction studies. In this paper, we introduce a new people-object interaction dataset that comprises 38 series of 30-view multi-person or single-person RGB-D video sequences, accompanied by camera parameters, foreground masks, SMPL models, some point clouds, and mesh files. Video sequences are captured by 30 Kinect Azures, uniformly surrounding the scene, each in 4 K resolution 25 FPS, and lasting for 1~19 seconds. Meanwhile, we evaluate some SOTA NVS models on our dataset to establish the NVS benchmarks. We hope our work can inspire further research in human-object interaction.

1. Introduction

Human-object interaction refers to the interactions of single or multiple people with objects and is a common type of scenario in everyday life [1]. In recent years, research on novel view synthesis (NVS) on human-object interaction has received much attention. NVS allows for the synthetic generation of a virtual view placed at an arbitrarily selected position in a three-dimensional scene. NVS in scenes of human-object interaction is necessary for high-level vision tasks like action analysis, visual scene answering, and video understanding. The main challenges are the complex interaction patterns and the severe occlusions [2].