Web Content Outlier Mining Through Using Web Datasets Research Paper

Web Content Outlier Mining Through Using Web Datasets - Research Paper Example The amount of knowledge sought by an individual is always very specific. Search of specific knowledge from the huge databases and data warehouses has become an essential need. Knowledge seekers while surfing web content on internet, come across large amount of information which is irrelevant to the subject of search and it is generally referred as web content outlier. This research investigates different methods of extracting outliers from web contents. Using web contents as data sets, it is aimed to find an algorithm which extract and mine varying contents of web documents of same category. Structure of HTML is used in this paper with various available techniques to model for mining web content outliers. Web content outlier’s mining using web datasets and finding outlier in them. In this modern time, the information is overloaded with huge databases, data warehouses and websites. The growth of internet and uploading and storing of information in bulk on websites is exponentia l. Accessibility of information is also made very easy for common man through internet and web-browser technology. The structure of web is global, dynamic, and enormous which has made it necessary to have tools for automated tracking and efficient analyzing of web data. This necessity of automated tools has started the development of systems for mining web contents. Extracting data is also referred as knowledge discovery in datasets. The process of discovering patterns which are interesting and useful and the procedures for analyzing and establishing their relationships are described as data mining. Most of the algorithms used today in data mining technology find patterns that are frequent and eliminate those which are rare. These rare patterns are described as noise, nuisance or outliers. (Data mining, 2011) The process of mining data involves three key steps of computation. First step is the process of model-learning. Second step is the model evaluation and the third step is the u se of the model. To clearly understand this division, it is necessary to classify data. (Data mining, 2011) The first step in data mining is the model learning. It is the process in which unique attributes are found about a group of data. The attributes classify the group and based on it an algorithm is built which defines the class of the group and establishes its relationship. Dataset with their attributes known are used to test this algorithm, generally called classifier. Results produced by the classifier assist in determining minimum requirements for accepting data of the known class. It gives the amount of accuracy of the model and if the accuracy is acceptable, the model is used to determine the similarity of each document or data in a dataset. (Data mining, 2011) The second step in data mining is the model evaluation. Techniques used for evaluating the model depend largely on the known attributes of data and knowledge types. The objectives of data users determine the tasks f or data mining and types of analysis. These tasks include Exploratory Data Analysis (EDA), Descriptive Modeling, Predictive Modeling, Discovering Patterns and Rules, and Retrieval by Content. Outliers are generally found through anomaly detection, which is to find instances of data that are unusual and unfit to the established pattern. (Data mining, 2011) Exploratory Data Analysis (EDA) show small data sets interactively and visually in the form of a pie chart or coxcomb plot. Descriptive Modeling is the technique that shows overall data distribution such as density estimation, cluster analysis and segmentation, and dependency modeling. Predictive Modeling uses variables having known values to predict the value of a single unknown variable. Classification