Loading [a11y]/accessibility-menu.js
Reading Users' Minds From Their Eyes: A Method for Implicit Image Annotation | IEEE Journals & Magazine | IEEE Xplore

Reading Users' Minds From Their Eyes: A Method for Implicit Image Annotation


Abstract:

This paper explores the possible solutions for image annotation and retrieval by implicitly monitoring user attention via eye-tracking. Features are extracted from the ga...Show More

Abstract:

This paper explores the possible solutions for image annotation and retrieval by implicitly monitoring user attention via eye-tracking. Features are extracted from the gaze trajectory of users examining sets of images to provide implicit information on the target template that guides visual attention. Our Gaze Inference System (GIS) is a fuzzy logic based framework that analyzes the gaze-movement features to assign a user interest level (UIL) from 0 to 1 to every image that appeared on the screen. Because some properties of the gaze features are unique for every user, our user adaptive framework builds a new processing system for every new user to achieve higher accuracy. The generated UILs can be used for image annotation purposes; however, the output of our system is not limited as it can be used also for retrieval or other scenarios. The developed framework produces promising and reliable UILs where approximately 53% of target images in the users' minds can be identified by the machine with an error of less than 20% and the top 10% of them with no error. We show in this paper that the existing information in gaze patterns can be employed to improve the machine's judgement of image content by assessment of human interest and attention to the objects inside virtual environments.
Published in: IEEE Transactions on Multimedia ( Volume: 14, Issue: 3, June 2012)
Page(s): 805 - 815
Date of Publication: 03 February 2012

ISSN Information:


I. Introduction

The history of eye-tracking goes back to the 19th century when scientists tried to study the reading process by direct observation [1], [2]. Along with the development of eye-tracking equipment, experimental studies in psychology and engineering have taken advantage of this new form of implicit human feedback opting for the day that every screen will have an affordable embedded eye-tracker. Some approaches to exploit this implicit feedback can be found in [3]–[8]. In this paper we discuss the use of eye-trackers for image annotation in the field of multimedia and vision to classify images as a function of the target template that guides user visual attention implicitly, promptly and accurately.

References

References is not available for this document.