Detecting Web Attacks Using Multi-stage Log Analysis

Web-based applications have gained universal acceptance in every sector of lives, including social, commercial, government, and academic communities. Even with the recent emergence of cloud technology, most of cloud applications are accessed and controlled through web interfaces. Web security has therefore continued to be fundamentally important and extremely challenging. One major security issue of web applications is SQL-injection attacks. Most existing solutions for detecting these attacks use log analysis, and employ either pattern matching or machine learning methods. Pattern matching methods can be effective, dynamic, they however cannot detect new kinds of attacks. Supervised machine learning methods can detect new attacks, yet they need to rely on an off-line training phase. This work proposes a multi-stage log analysis architecture, which combines both pattern matching and supervised machine learning methods. It uses logs generated by the application during attacks to effectively detect attacks and to help preventing future attacks. The architecture is described in detail, a proof-of-concept prototype is implemented and hosted on Amazon AWS, using Kibana for pattern matching and Bayes Net for machine learning. It is evaluated on 10,000 logs for detecting SQL injection attacks. Experiment results show that the two-stage system has combined the advantages of both systems, and has substantially improved the detection accuracy. The proposed work is significant in advancing web securities, while the multi-stage log analysis concept would be highly applicable to many intrusion detection applications.