An Image Retrieval System Based on Automatic Image Annotation to Facilitate Digital Humanities Research

In addition to digital text, the digital image has become an important data type used to support digital humanities research. An image retrieval system based on automatic image annotation (IRS-AIA) is therefore developed based on Mask R-CNN to enhance image retrieval performance through enriching the metadata of an image by the recognized object labels for digital humanities research. In contrast, the metadata of an image manually determined by a human is called subjective labels. To verify whether IRS-AIA can assist digital humanists in the effectiveness of image interpretation, counterbalanced design in quasi-experimental research was applied. A total of 14 users were divided into two groups to complete the designed image interpreting tasks by operating IRS-AIA and general image retrieval tool (GIRT) through crossing the use of two systems. Analytical results show that IRS-AIA could effectively assist users in interpreting image situations. The use of IRS-AIA could acquire better image retrieval precision and recall rates than GIRT. However, the image interpretation effectiveness between the use of GIRT and IRS-AIA does not achieve a statistically significant difference. Subject label and object label present distinct purposes in image retrieval and both labels could satisfy users' needs for different information retrieval ways.