Evaluating the retrieval effectiveness of search engines using Persian navigational queries

In this paper, we evaluate search engines' performance by considering how they behave for Persian users' navigational queries. About 2000 queries were posed to three search services supporting Persian language (Google and Bing as well-known services and Parsijoo as a local search service). All queries were of navigational type and in Persian language. These queries were created in a way to have web pages from Iranian web sites as their targets. After sending these queries to search engines the position of the target web pages in the result list of search engines were recorded. Then mean reciprocal rank and Success N measures were used for evaluation of the search services' effectiveness. At the end, we find out that although some search services have better performance for such kind of queries, but from a statistical point of view their efficiency is near to each other. For example for the MRR measure Parsijoo score is 0.58 and Google score is 0.54 and Bing Score is 0.52.