An Improved Approach to Bengali Keyphrase Extraction

This paper presents a new approach for automatically extracting key phrases from a Bengali document. Our proposed approach presented in this paper has two important steps: (1) a shallow parsing based candidate key phrase identification that uses lexical information and case markers for candidate key phrase identification and (2) choosing the best items from the set of the candidates using a ranking method that combines the statistical features and the linguistic features for ranking the candidates. The feature set includes term frequency, position of the phrase's first occurrence, named entity information and lexical information. The proposed system has been tested on a collection of Bengali news documents. The experimental results show that it performs better than the existing approaches to which it is compared.

I. Introduction

A keyphrase is an important phrase consisting of sequence of words. A set of keyphrases capture the main topics covered in a document. It is very useful in automatic indexing, text document classification, document clustering, summarization [1] [2], etc. Keyphrases are meant to serve various goals. For example, 1) when they are printed on the first page of a journal document, the goal is summarization. They enable the reader to quickly determine whether the given article worth in-depth reading. 2) When they are added to the cumulative index for a journal, the goal is indexing. 3) When a search engine form contains a field labeled keywords, the goal is to enable the reader to make the search more precise.