Heuristic Phishing Detection and URL Checking Methodology Based on Scraping and Web Crawling

With the improvement of techniques used by the attackers, the detection and prevention of threats such as phishing and malware can represent a problem and computational challenge. Recent research has found that phishing and malicious code infection are the main threats triggered by social engineering. In this work, the attack vectors that cause these threats are analyzed, proposing a method of checking specific strings in URLs and e-mail messages, which can be used in conjunction with proxies and Anti Spam filters. The method was implemented in an experimental scenario and is capable of detecting the presence of the main elements that have direct contact with the user, such as: form fields, redirection of links and downloadable files. Furthermore, the proposed method was able to detect phishing URLs with accuracy values between 73.3% and 97.66% with an average time of 30 seconds.