1. Introduction
A central goal in computer vision is to develop models that can understand diverse and novel scenarios in the visual world. While this has been difficult for methods developed on datasets with closed label spaces, web-scale image-text pretraining has recently led to dramatic improvements in open-world performance on a range of image-level vision tasks [12], [26], [19].