Wrapper (data mining)
Wrapper in data mining is a program that extracts content of a particular information source and translates it into a relational form.[1] Many web pages present structured data - telephone directories, product catalogs, etc. formatted for human browsing using HTML language. Structured data are typically descriptions of objects retrieved from underlying databases and displayed in Web pages following some fixed templates. Software systems using such resources must translate HTML content into a relational form. Wrappers are commonly used as such translators. Formally, a wrapper is a function from a page to the set of tuples it contains.
Wrapper generation[edit]
There are two main approaches to wrapper generation: wrapper induction and automated data extraction. Wrapper induction uses supervised learning to learn data extraction rules from manually labeled training examples. The disadvantages of wrapper induction are
- the time-consuming manual labeling process and
- the difficulty of wrapper maintenance.
Due to the manual labeling effort, it is hard to extract data from a large number of sites as each site has its own templates and requires separate manual labeling for wrapper learning. Wrapper maintenance is also a major issue because whenever a site changes the wrappers built for the site become obsolete. Due to these shortcomings, researchers have studied automated wrapper generation using unsupervised pattern mining. Automated extraction is possible because most Web data objects follow fixed templates. Discovering such templates or patterns enables the system to perform extraction automatically.[2] More recently, the increasing availability of Linked Data has enabled methods[3] that can automatically learn and maintain wrappers using such resources, based on the principle of 'distant supervision' [4]. In this case, sample instances of concepts are firstly collected from publicly available Linked Datasets. These are then searched within a collection of Webpages and their matched occurrences are annotated. Although these annotations can be noisy, they prove to be useful training data for learning Webpage wrappers.
Wrapper generation on the Web is an important problem with a wide range of applications. Extraction of such data enables one to integrate data/information from multiple Web sites to provide value-added services, e.g., comparative shopping, object search, and information integration. –the wrapper content can be enhanced
See also[edit]
- Business intelligence (section Semi-structured or unstructured data)
- Web scraping
Sources[edit]
- ^ Nicholas Kushmerick, Daniel S. Weld, Robert Doorenbos, Wrapper Induction for Information Extraction Proceedings of the International Joint Conference on Artificial Intelligence, 1997
- ^ Liu, B. Web Data Mining: Exploring Hyperlinks, Contents and Usage Data, Springer, 2007.
- ^ Anna Lisa Gentile, Ziqi Zhang, Isabelle Augenstein, and Fabio Ciravegna. 2013. Unsupervised wrapper induction using linked data. In Proceedings of the seventh international conference on Knowledge capture (K-CAP '13). ACM, New York, NY, USA, 41-48. DOI: https://dx.doi.org/10.1145/2479832.2479845
- ^ Mike Mintz, Steven Bills, Rion Snow, and Dan Jurafsky. 2009. Distant supervision for relation extraction without labeled data. In Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP: Volume 2 - Volume 2 (ACL '09), Vol. 2. Association for Computational Linguistics, Stroudsburg, PA, USA, 1003-1011
