Article Detection on the Web using DOM and text based features

Web pages are constantly getting flooded with distractive content like ads, popups, etc. which take away the focus from the main article of the page. Compared to desktops, smartphones-whose usage for browsing online has been on a steep upward trend-have a smaller screen size thereby compounding the problem. To give the user a good browsing experience on phone and hence increase their engagement with the browser, it is imperative to provide a clutter-free experience to him. Existing techniques do this in two phases-phase one detects if there is an article on a given Web page and phase two proceeds to extract the article in order to display it, removing unwanted elements of the Web page. In this paper, we propose to use a random forest model for article detection in Web pages using DOM and text-based features. With this model, we were able to achieve a precision of 0.91 and a recall of 0.92 which beats the state of the art techniques which are being used by major Android browsers in the market today.