ICDAR 2005 text locating competition results

This paper describes the results of the ICDAR 2005 competition for locating text in camera captured scenes. For this we used the same data as the ICDAR 2003 competition, which has been kept private until now. This allows a direct comparison with the 2003 entries. The main result is that the leading 2005 entry has improved significantly on the leading 2003 entry, with an increase in average f-score from 0.5 to 0.62, where the f-score is the same adapted information retrieval measure used for the 2003 competition. The paper also discusses the Web-based deployment and evaluation of text locating systems, and one of the leading entries has now been deployed in this way. This mode of usage could lead to more complete and more immediate knowledge of the strengths and weaknesses of each newly developed system.

1 Introduction

In recent years there has been some significant research into these general reading systems that are able to locate and/or read text in scene images [10], [2], [9]. As with all complex pattern recognition tasks, it is essential to quote results on standard datasets in order to have meaningful evaluation. The first publicly available ground-truthed dataset on which to evaluate such systems was that used for the IC-DAR 2003 robust reading competitions. The test data for those competitions was kept private, and is used to assess the 2005 entries. The test dataset consists of 501 images captured with a variety of digital cameras. Cameras were used with a range of resolution and other settings, with and without flash, with the particular settings chosen at the discretion of the photographer. The images include household objects, road signs, shop signs, bill-boards and posters, and book covers. They span a wide range of apparent difficulties. A training dataset of 500 images of a broadly similar nature was made publicly available in Autumn 2002. Entrants were also free to tune their systems on their own data.