System Analysis and Error Detection: An approach towards server monitoring using Natural Language Processing

Advanced technologies bring advancement and benefits to the majority of fields. Different applications have been developed in areas related to machine learning and data mining. But our server subsystems still require more of human intervention to analyze things. Hence an approach has been discussed to make the primitive servers as a smart server with the help of data analysis and machine learning techniques. There are various tools which can monitor a number of systems with automation, but automation cannot give answers like with what kind of errors my system got impacted more? How my system performs under load and without load? All these answers can be achieved through system analysis. Hence, the term “analysis” can be described as the process of studying complex structures to get meaningful results in order to build system that are smart and robust. In the era of data explosion, where a system is much complex now, a tool would be needed with which can compute such queries Although for analysis, various tools and techniques exists but their schema could not handle the big data or large number of data and automation scripts cannot provide optimized results hence there is a huge need to have modern techniques for better optimization. The automated results are not reliable enough as it still requires more of expert knowledge to analyze it further. In this paper, the focus with term “Analysis” is in point of view to check the behavior of the system parameter i.e., CPU utilization through visualization in real time and automatic error parsing and detection of error is taken under consideration so that a domain expert is no longer required to solve our problem instead the server admin person can analyze own.

I. Introduction

With the advancement and complexity in the IT industry and the age of data explosion, debugging becomes tougher. As verification engineer can't check each task output generated by automation scripts and that is not reliable. So, there is a huge need to find alternatives that aim to find fault in more ease manner. Needs have been specified for three reasons: (1) For system analysis, either need to depend on the expert or automation script but training people to become such expert involves cost and is inefficient instead the machine can be trained to learn itself. Machine learning can handle large data and generalize results from learning patterns on a set of training data set. The Natural Language Processing can automate the process of log analysis and it does not rely on explicit programming means it does not require to know all truth about the problem. Depending on the type of data available machine learning technique can be chosen for text mining and anomaly detection.