Object Permanence in Videos: DNN Performance vs. Human Ability

Object permanence, the idea that an object still exists when not in view, is a fundamental aspect of human cognition that typically develops in infancy. However, object permanence is often left out of machine vision models designed to identify objects in video. As machine vision technology becomes more widespread, it is important to understand the key differences between machine and human visual processing, and how we can bridge these gaps. In order to compare object permanence capabilities in humans and machine models, we created a custom video dataset of objects undergoing occlusion, including frames with partial and full occlusions. Human object detection performance on the dataset was assessed via survey, and compared with the SOTA object detection algorithm YOLOv3, as well as a novel recurrence-based model we call YOLO-LSTM. Although human performance degrades significantly when identifying object size and location under occlusion, human subjects almost always recognize that occluded objects still exist, while YOLOv3 shows linearly decreasing performance as objects become more occluded. YOLO-LSTM has a lower accuracy overall than YOLO-v3, but demonstrates more humanlike behavior.