Data structuring and effective retrieval in the mining of web sequential characteristic

The Web data mining based on sequential characteristics is a mining technology focusing on text data on Web pages and link structure and combing sequential characteristics on the basis of the mining of Web structure and Web contents. A huge number of data information is carried on Web, and it is increased at a geometric speed every day. As time goes by, the effectiveness of a great number of data is continuously reduced, and they even become completely useless. How to clean these useless data, find out hidden regular contents among a great number of data, and solve the quality problem of data application has become the research hotspot in the Web data mining technology at present. All the information objects on Web can be generally divided into two categories: Structured data and semi-structured data. Those that can be expressed in database structure are called structured data; those expressed in various forms with text as representative are called semi-structured data. The greatest feature of Web data is semi-structuring. Such kind of semi-structured data are relevant to time sequence, meanwhile, time effect of data is also related to time sequence. In the article, discussion is made about how to use the sequential characteristic in the course of Web data mining to carry out structural transfer of semi-structured data based on time effect of data, that is the structuring of Web data, and solve the problem about effectiveness in retrieval accordingly.

I. Web Data Environment

From the perspective of study on data warehouse, information on websites can be also regarded as a bigger and more complicated data warehouse. Every site on Web is a data source, and every data source is heterogeneous, for the information and tissue of every site is different. From the perspective of multidimensional database, the Web data environment is a huge heterogeneous data warehouse environment constituted based on two-dimensional plane data together with time dimension and link dimension. The Web data mining technology is much more complicated than the data mining focusing on conventional data warehouse. Therefore, the integration of heterogeneous data among the sites must be studied. Only the data at these sites are integrated, and a uniform view is provided for users, can it be possible to obtain what it is needed from the huge data resources.