Similarity based Dynamic Web Data Extraction and Integration System from Search Engine Result Pages for Web Content Mining

Short Paper
ACEEE Int. J. on Information Technology, Vol. 3, No. 1, March 2013

Similarity based Dynamic Web Data Extraction and
Integration System from Search Engine Result Pages
for Web Content Mining
Srikantaiah K C1, Suraj M1, Venugopal K R1, and L M Patnaik2
1
Department of Computer Science and Engineering,
University Visvesvaraya College of Engineering, Bangalore University, Bangalore , India
srikantaiahkc@gmail.com
2
Honorary Professor, Indian Institute of Science, Bangalore, India

Abstract— There is an explosive growth of information in available today for the purpose of data mining. Web data
the World Wide Web thus posing a challenge to Web users Extraction is the process of extracting the information that users
to extract essential knowledge from the Web. Search are interested in, from Semi-structured or unstructured web
engines help us to narrow down the search in the form of pages and saving the information as the XML document or
Search Engine Result Pages (SERP). Web Content Mining
relationship model. During Web data extraction phase Search
is one of the techniques that help users to extract useful
information from these SERPs. In this paper, we propose Engine Result Pages are crawled and stored in the local
two similarity based mechanisms; WDES, to extract desired repository. Web database Integration is a process of extracting
SERPs and store them in the local depository for offline the required data from the web pages stored in the Local
browsing and WDICS, to integrate the requested contents repository and Integrates the extracted data and stored in the
and enable the user to perform the intended analysis and database.
extract the desired information. Our experimental results
show that WDES and WDICS outperform DEPTA [1] in A. Motivation
terms of Precision and Recall. In the recent years there has been lot of improvements on
technology with products differing in the slightest of terms.
Index Terms— Offline Browsing, Web Data Extraction, Web Every product needs to be tested thoroughly and internet plays
Data Integration, World Wide Web, Web Wrapper a vital role in gathering information for the effective analysis of
the products. In our approach, we replicate search engine result
I. INTRODUCTION pages locally based on comparing page URLs with a predefined
The World Wide Web (WWW) has now become the threshold. The replication is such that the pages are accessible
largest knowledge base in the human history. The Web locally in the same manner as on the web. In order to make the
encourages decentralized authorizing in which users can data available locally to the user for analysis we extract and
create or modify documents locally, which makes integrate the data based on the prerequisites which are defined
information publishing more convenient and faster than in the configuration file.
ever. Because of these characteristics, the Internet has B. Contribution
grown rapidly, which creates a new and huge media for
information sharing and exchange. Most of the information In a given set of web pages, it is difficult to extract matching
on the Internet cannot be directly accessed via the static data. So, we have to develop a tool that is capable of extracting
link, must use Keywords and Search Engine. Web search the exact data from the web pages. In this paper, we have
engines are programs used to search information on the developed WDES algorithm, which provides offline browsing
WWW and FTP servers and to check the accuracy of the of the pages. Here, we integrate the downloaded content onto a
data automatically. When searching for a topic in the defined database and provide a platform for efficient mining of
WWW, it returns many links or web sites related i.e., the data required.
Search Engine Result Pages (SERP) on the browser to a C. Organization
given topic. Some data in the internet is visible to search The rest of this paper is organized as follows: Section II
engine is called surface web, where as some data such as describes algorithms related to web data extraction, integration
dynamic data in dynamic database is invisible to search and crawling, Section III defines the problem, describes
engine is called deep web.There are situations in which mathematical model and algorithm, Section IV describes the
the user needs those web pages on the Internet to be system architecture, Section V comprises of experimental results
available offline for convenience. The reason being offline and analysis. The concluding remarks are summarized in section
availability of data, limited download slots, storing data VI.
for future use, etc.. This essentially leads to downloading
raw data from the web pages on the Internet which is a
major set of the inputs to a variety of software that are

© 2013 ACEEE 42
DOI: 01.IJIT.3.1. 1102

Short Paper

II. RELATED WORK template extraction and reconstruction is depicted in an order
of how the data is been parsed on the web page and in
Zhai et al., [1] propose an algorithm DEPTA for the
reconstructing the page to overcome the noise in the page. It
structured data extraction from the web based on partial tree
does not parse through the dynamic content of scripts on
alignment, studying the problem structured data extraction
the page.Liu et al., [6] propose a method to extract main text
from arbitrary web pages. The main objective is to
from the body of a page based on the position of DIV. It
automatically segment data records in a page, extract data
reconstructs and analyzes DIV in a web page by completely
items/fields from these records and store the extracted data
simulating its display in browser, without additional
in a database. It consists of two steps, i.e identifying
information such as web page template and its implementation
individual records in a page and aligning and extracting data
complexity is quite low. The core idea includes the concept
items from the identified records, using visual information
of atomic DIV, i.e., a DIV block that does not include other
and tree matching and a novel partial alignment technique
DIVs. Then filter out on redundant, reconstruct, filter invalids,
respectively.
clustering, reposition analysis and finally storing the elements
Ananthanarayanan et al., [2] propose a method for offline
of data in an array. The selected DIVs in selected array contain
web browsing that is minimally dependent on the real time
the main text of the page. We can get the main text by
network availability. The approach defined is to make use of
combining these DIVs. The method has a high versatility
the Really Simple Syndication (RSS) feeds from web servers
and accuracy. The majority of the web page content is been
and pre-fetch all new content specified, defining the content
made up of tables and this approach does not address the
section of the home page. It features intelligent pre-fetching,
table layout data. This drastically reduces the accuracy of
robust and resilient measures for intermittent network
the entire system.
handling, template identifier and local stitching of the
Dalvi et al., [7] explore a novel approach based on temporal
dynamic content into the template. It does not provide all the
snapshots of web pages to develop a tree-edit model of
information in the page. Also, the content defined in the RSS
HTML and use this model to improve wrapper construction.
feeds may not be updated nor they do provide for dynamic
The model is attractive in that the probability that a source
changes in the page.
tree has evolved into a target tree can be estimated efficiently,
Myllymaki et al., [3] describe ANDES, a software framework
in quadratic time in the size of the trees, making it a potentially
that provides a platform for building a production-quality
useful tool for a variety of tree-evolution problems. An
Web data extraction process. Key aspects are that it uses
improvement on the robustness and performance and ways
XML technology for data extraction, including XHTML and
to prune the trees without hampering model quality is to be
XSLT and provides access to the deep Web. It addresses the
dealt with.Novotny et al., [8] represent a chain of techniques
issues of website navigation, data extraction, structure
for extraction of object attribute data from web pages. They
synthesis, data mapping and data integration. The framework
discover data regions containing multiple data records and
shows that production-quality web data extraction is quite
also provide for a detailed algorithm for detail page extraction
feasible and that incorporating domain knowledge into the
based on the comparison of two html sub trees. They describe
data extraction process can be effective in ensuring the high
the extraction from two kinds of web pages: master pages
quality of extracted data. Data validation technique, a cross
containing structured data about multiple objects and detail
system support, and a well established framework that could
pages containing data about single product respectively.
easily be made use of by any application are not discussed.
They combine the techniques of the master page extraction
Yang et al., [4] propose a novel model to extract data from
algorithm, detail page extraction algorithm and comparison
deep web pages. It four layers, among which the access
of sources of two web pages. The approach makes use of the
schedule, extraction layer and data cleaner are based on the
selection of attribute values based on the Document Object
rules of structure, logic and application. The model first uses
Model (DOM) structure described in the web pages. It has
the two regularities of the domain knowledge and interface
better precision of extraction of values from pages defined in
similarity to assign the tasks that are proposed from users
the master and detailed format and also having to minimize
and chooses the most effective set of sites to visit. Then, the
the user effort to the core. Enhances to the approach may
model extracts and corrects the data based on logical rules
include the page level complexity of having multiple
and structural characteristics of acquired pages. Finally, it
interleaved detail pages to be traversed, coagulation of
cleans and orders the raw data set to adapt the customs of
different segments in the master page and series
application layer for later use by its interface.
implementation of pages.
Yin et al., [5] propose web page templates and DOM
Nie et al., [9] provide an approach of obtaining data from
technology to effectively extract simple structured information
the result set pages by crawling through the pages for data
from the web. The main contents include the methods based
extraction based on the classification of the URL (Unified
on edit distance, DOM document similarity judgement,
Resource Locator). It extracts data from the pages by
clustering methods of web page templates and programming
computing the similarity between the URL’s of hyperlinks
an information extraction engine. The method provides on
and classifying them into four categories, where each
the steps for information extraction DOM tree parsing and
category maps to a set of similar web pages which separate
on how a page similarity judgement is to be made. The
result pages from others. Then makes use of the page probing
© 2013 ACEEE 43
DOI: 01.IJIT.3.1. 1102

Short Paper

method to verify the correctness of classification and improve evaluating page relevance can be anything that outputs a
the accuracy of crawled pages. The approach makes use of binary classification. For each retrieved page, the apprentice
the minimum edit distance algorithm and URL-field algorithm is trained on information from baseline classifier and features
to calculate the similarity between URLs of hyperlinks around the link extracted from the parent page to predict the
respectively. However there are a few constraints to this relevance of the page pointed to by the link. Those
approach. It is not able to resolve issues of pages related to predictions are then used to order URLs in the crawl priority
the partial page refreshments by the use of javascript queue. Number of false positives has decreased significantly.
engines.Papadakis et al., [10] describe STAVIES, a novel Ehrig et al., [16] propose an ontology-based algorithm for
system for information extraction from web data source page relevance computation which is used for web data
through automatic wrapper generation using clustering extraction. After preprocessing, words occurring in the
technique. The approach is to exploit the format of the Web ontology are extracted from the page and counted. Relevance
pages to discover the underlying structure in order to finally of the page with regard to user selected entities of interest is
infer and extract pieces of information from the web page. then computed by using several measures such as direct
The system can operate without human intervention and does match, taxonomic and more complex relationships on ontology
not require any training. graph. The harvest rate of this approach is better than baseline
Chang et al., [11] survey the major web data extraction focused crawler.Srikantaiah et al., [17] propose an algorithm
systems and compare them in three dimensions: the task for web data extraction and integration based on URL
domain, the automation degree and the techniques used. similarity and Cosine Similarity. Extraction algorithm is used
These approaches emphasize on availability of robust, flexible to crawl the relevant pages and stores in local repository.
Information Extraction (IE) systems that transform the web Integration algorithm is used to integrate the similar data in
pages into program-friendly structures such as a relational various records based on cosine similarity.
database becomes a great necessity. The paper mainly
focuses on the IE from semi structured documents and III. PROPOSED MODEL AND ALGORITHMS
discusses only those that have been used for web data. Based
A. Problem Definition
on the survey it makes many points such as the trend of
developing highly automatic IE systems, which saves not Given a start page URL and a configuration file, the main
only the effort for programming, but also the effort for labeling, objective is to extract pages which are hyperlinked from the
enhancements for applying the techniques to non-html start page and integrate the required data for analysis using
documents such as medical records and curriculum vitae to data mining techniques. The user has sufficient space on the
facilitate the maintenance of larger semi structured machine to store the data that is downloaded. The basic
documents. notations used in the model are shown in Table 1.
Angkawattanawit et al., [12] propose an algorithm to TABLE I. BASIC N OTATIONS
improve harvest rate by utilizing several databases like seed
URLs, topic keywords and URL relevance predictors that are
built from previous crawl logs. Seed URLs are computed using
BHITS [13] algorithm on previously found pages by selecting
pages with high hub and authority scores that will be used
for future recrawls. The interested Keywords for the target
topic are extracted from anchor tags and title of previously
found relevant pages. Link crawl priority is computed as a B. Mathematical Model
weighted combination of popularity of the source page,
similarity of link anchor text to topic keywords and predicted Web Data Extraction using Similarity Function (WDES):
link score which is based on previously seen relevance for A connection is been established to the given URL S and the
that specific URL.Aggarwal et al., [14] propose an approach page is processed with the parameters obtained from the
to crawl the interested web pages using the concept of configuration file C. On completion of this, we obtain the
“intelligent crawling”. In this concept the user can specify web document that contains the links to all the desired
an arbitrary predicate such as keywords, document similarity, contents that are obtained out of the search performed. The
etc., which are used to determine documents relevance to the web document contains individual sets of links that are
crawl and the system adapts itself in order to maximize the displayed on each of the search results pages that are
harvest rate. A probabilistic model for URL priority prediction obtained. For example, if a search result obtained contains
150 records displayed as 10 records per page (in total 15 pages
is trained using URL tokens, information about content of in-
of information), we would have 15 sets of web documents
linking pages, number of sibling pages matching the predicate
each containing 10 hyperlinks pointing to the required data.
so far and short-range locality information Chakrabarti et al.,
This forms the set of web documents, W. i.e.,
[15] propose models for finding URL visit priorities and page
relevance. The model for URL ranking called “apprentice” is W  {wi : 1  i  n} . (1)
on-line trained by samples consisting of source page features Each web document wi W is read through to collect the
and the relevance of the target page but the model for hyperlinks that are contained in it, that are to be fetched to
© 2013 ACEEE 44
DOI: 01.IJIT.3.1.1102

Short Paper

obtain the data values. We, represent this hyperlink set as integrate data in Depth First Search (DFS) manner. Hence
H(W). Thus, we consider H(W) as a whole set containing all their complexity is O(n2), where n is the number of hyperlinks
the sets of hyperlinks on each page wi W. i.e., in H(W).
H (W )  {H ( wi ) : 1  i  n} . (2)
TABLE II. ALGORITHM: WEB D ATA EXTRACTION USING SIMILARITY (WDES)
Then, considering each hyperlink hj H(wi), we find the
similarity between hj and S, using (3)
min( nf ( hj ), nf ( S ))

 fsim( f h , f S ) i j i

SIM (hj, S )  i 1 (3)
(nf (hj )  nf ( S )) / 2
where nf(X) is the number of fields in X and fsim(fihj, fiS) is
defined as
(4)
The similarity SIM(hj ,S) is the value that lies between 0
and 1, this value is used to compare with the defined threshold
To (0.25), we download the page corresponding to hj to local
repository if SIM(hj , S) To . The detailed algorithm of
WDES is given in Table 2.The algorithm WDES navigates
the search result page from the given URL S and configuration
file C and generates a set of web documents W. Next, call the
function Hypcollection to collect hyperlinks of all pages in
wi, indexed by H(wi), page corresponding to H(wi) is stored
in the local repository. The function webextract is recursively
called for each H(wi). Then, for each hi H(wi), similarity
between hi and S is calculated using (3), if SIM(hi,S) is greater
than the threshold To, then page corresponding to hi is stored
and collect all the hyperlinks in hi to X. Continue this process
for X, until it reaches maximum depth l.
Web Data Integration using Cosine Similarity(WDICS):
The aim of this algorithm is to extract data from the
downloaded web pages (those web pages that are available
in the local repository i.e., output of WDES algorithm) into
the database based on attributes and keywords from the
configuration file Ci. We collect all result pages W from local
repository indexed by S, then H(W) is obtained by collecting
all hyperlinks from W, considering each hyperlink hj H(wi)
such that k keywords in Ci. On existence of k in hj, we IV. SYSTEM ARCHITECTURE
populate the new record set N[m, n] by passing page hj and
obtaining values defined with respect to the attributes[n] in The entire system has been divided into three blocks that
Ci. We then populate the old record set O[m, n] by obtaining are able to accomplish the dedicated tasks, working
all values with respect to attributes[n] in database. For each independently from one another. The first block namely, the
record i, 1 i m we find the similarity between N[i] and Web Data Extractor connects to the internet to extract the
O[i] using cosine similarity, web pages described by the user and stores it onto a local
n
repository. Also, it stores the data in a format that is likely to
N O
j 1
ij ij
be an offline browsing system. Offline Browsing means that
Sim Re cord ( Ni, Oi )  the user is able to browse through the pages that are been
n n
 j 1
N 2ij  j 1O 2ij downloaded, by the use of a web browser without having to
(5) connect to the internet. Thus, it would be convenient for the
If similarity between records is equal to zero, then we compare user to go through the pages as and when he needs.The
each attribute[j] 1 j n in the records and form second block namely, the Database Integrator extracts the
IntegratedData with use of Union operation and store in the vital piece of information that the user needs from the
database. downloaded web pages and creates a database with the set
IntegratedData = Union(Nij ,Oij ). (6) of tables and respective attributes in the database. These
The detailed algorithm of WDICS is shown in Table 3. tables are populated with the data extracted from the locally
The algorithms WDES and WDICS respectively extract and downloaded web pages. This makes it easy for the user to
© 2013 ACEEE 45
DOI: 01.IJIT.3.1. 1102

Short Paper

TABLE III. ALGORITHM: WEB D ATA INTEGRATION USING COSINE SIMILARITY pre-defined in the set of attributes, together with the tables.
(WDICS) This block needs the presence of the usage attributes that
are extracted from the downloaded web content. The usage
attributes are mainly defined in a file termed as configuration
file that the user needs to prepare before the execution. The
file also contains the table to which the data extracted are to
be added together with the table attributes and mapping.
On successful entries of the input, the integration block
is able to accomplish the task of extracting the content from
the web pages. Since, the web pages do tend to remain in the
same format as on the internet, it is easy to be able to navigate
across these pages as the links refer to the locally extracted
pages. The extracted content is then processed to meet the
desired data attributes that are listed on the database and the
values are dumped onto it. This essentially creates rows of
extracted information in the database. The outcome of second
block results in tuples in the database that are essentially
extracted out from the extracted pages got from first block.
Now that the database is been formed with the vital
information contained in it, it would well be the task of the
Analyzer, i.e., the third block referred as GUI to provide the
user with the functionality on how to deal with the contained
data. The GUI provides for the interfaces that the user is able
to achieve so as to obtain the data contained in the database,
as and how he needs it. It acts as a miner that provides the
user with the information obtained from the result of queries
that are defined. The GUI also has options on referring the
other blocks as requested by the user. This indeed interfaces
all the three blocks, thereby providing the user with a better
understanding and handling feature.
It is quite essential to know a few things that relate to the
working of the entire system. First of all, it is very much
query out his needs from the database. The overall view of essential that the inputs, initializations and the pre-requisite
the system is as shown in Fig. 1. The system initializes on a data such as the usage attributes and/or configuration files
set of inputs for each block. These blocks are highlighted that are been defined, do fall in a particular order and stick to
and framed according to the flow of data. The inputs that are the conventions defined on them. It is important to have the
needed for the start of the first block, i.e., the Web Data blocks to perform the tasks in order. The next thing is that,
Extractor are fed in through the initialize phase. On receiving although the blocks do persist to function independently it
the inputs the search engine performs its task of navigating is important that without the essential requirements, it is of
to the page on the given URL and entering the criteria defined no use to extract its functionality. Unless the pre-requisites
on the configuration file. This will populate a page from which of the blocks are not been met the functionality that they
the actual download can start. accomplish is of no use. Similarly, to query out the needs of
The process of extraction is to achieve the task of the user through the help of the GUI, it is essential that the
downloading the contents from the web and to store them data is available in the database. It is also essential that the
onto the local repository. Then onwards the process of first block and second block be run often and in unison so as
extracting the pages iterates over, resulting in the outcome of to have an update on the current and on-going dataset.
offline web pages (termed extracted pages). The extracted
pages are available offline on a pre-described local repository. IV. EXPERIMENTAL RESULTS
The pages in the repository can be browsed through in the The algorithms are independent of usage and that they
same manner as that available on the internet with the help of could be used on any given platform and dataset. The
a web browser. The noticeable thing here is that the pages experiment was conducted on the Bluetooth SIG Website
are available offline, thereby are much faster to be accessed. [18], The Bluetooth SIG (Special Interest Group) is a body
The outcome of first block is to be chained with other that oversees the licensing and qualification of Bluetooth
attributes that are essential for the second block namely, the enabled product. It maintains the database of all the qualified
Database Integrator. The task of second block is to extract products in a database which can be accessed through queries
the vital information from the extracted pages, process it, and on its website and hence is a domain specific search engine.
store it in accordance onto the database. The database is
© 2013 ACEEE 46
DOI: 01.IJIT.3.1. 1102

Short Paper

Fig. 1. System Architecture
The qualification data contains a lot of parameters including involved in them. The EPL are those that are formed in unison
company name, product name, any previous qualification of the PRD’s. Each PRD is identified by a QDID (an id for
used, etc.. The main tasks for the Bluetooth SIG are to publish uniqueness) and each EPL is identified by the Model. Each
Bluetooth specifications, administer the qualification PRD may have many numbers of related EPL’s and each EPL
program, protect the Bluetooth trademarks and evangelize may have many numbers of related PRD’s.
Bluetooth wireless technology. The key idea behind the The listings as shown in Fig. 2, contain links which navigate
approach is to collect and automate the collection of to the detailed content of the products that are been displayed
competitive and market information from the Bluetooth SIG here. The navigations on the page reach to N number of
site. pages before the case of termination. Thereby we may want
The site contains data in the form of list that is displayed to parse across the links to reach all the places that the data
on the start-up page. The display is formed based on the we require is obtained. Here, although we have all the data
three types of listings; PRD 1.0, PRD 2.0 and EPL. The PRD being displayed on the web site, it is still not possible for us
1.0 and PRD 2.0 are the qualified products and design list and to prepare an analysis report based on the same. We want to
EPL are the end product list being displayed. Each of the play around with the data to get it to the form that we deserve
PRD’s listed here are products that contain the specifications it to be. Thereby, we incorporate the use of our algorithms to

Fig. 2. SERP in Bluetooth Sig Website

© 2013 ACEEE 47
DOI: 01.IJIT.3.1.1102

Short Paper

TABLE IV. PERFORMANCE EVALUATION B ETWEEN WDICS AND DEPTA

Fig. 4. Recall Vs. Attributes
Fig. 3. Precision Vs. Attributes
isms; WDES, to extract desired SERPs and store them in the
extract, integrate and mine the data from this site. The
local depository for offline browsing and WDICS, to integrate
experimental setup involves a Pentium Core 2 Duo
the requested contents and enable the user to perform the
machine with 2 GB RAM running windows. The algorithms
intended analysis and extract the desired information. This
have been implemented using a Java JDK and JRE 1.6, Java
results in faster data processing and effective offline browsing
Parser with an access to a SQLite database and active
for saving time and resources. Our experimental results show
broadband connection. Data has been collected from
that WDES and WDICS outperform DEPTA in terms of
www.bluetooth.org, which lists the qualified products of
Precision and Recall. Further, different Web mining techniques
Bluetooth devices. We have extracted the pages with dates
such as classification and clustering can be associated with
ranging from October 2005 to June 2011, all of which make up
our approach to utilize the integrated results more efficiently.
92 pages, with each page containing 200 records of
information and data extraction was possible from each of
REFERENCES
these pages. Hence, we have a cumulative set of data for
comparison based on the data extracted on the given attribute [1] Yanhong Zhai and Bing Liu, “Structured Data extraction from
mentioned in the configuration file. Precision is defined as the Web Based on Partial Tree Alignment,” IEEE Trans. on
the ratio of correct pages and extracted pages and recall is KDE, vol. 18, no. 12, pp. 1614-1627, 2006.
defined as the ratio of extracted pages and total number of [2] Ganesh Ananthanarayanan, Sean Blagsvedt, and Kentaro
Toyama, “OWeB: A Framework for Offline Web Browsing,”
pages. They are calculated based on the records extracted
IEEE Computer Society Washington, Proceedings of the Fourth
by our model, the records found by the search engine and Latin American Web Congress,pp. 15-24, 2006.
the total available records in the Bluetooth website. For [3] Jussi Myllymaki, “Effective Web Data Extraction with
different attributes as shown in the Table 4, the Recall and Standards XML Technologies,” Proceedings of the 10 th
Precision are calculated and their comparisons are as shown International Conference on World Wide Web, pp. 689-696,
in Figures 3 and 4 respectively. It is observed that the May 2001.
Precision of WDICS increases by 4% and the Recall increases [4] Jufeng Yang, Guangshun Shi, Yan Zheng, and Qingren Wang,
by 2% compared to DEPTA. Therefore, WDICS is more “Data Extraction from Deep Web Pages,” IEEE International
efficient than DEPTA because when an object is dissimilar to Conference on Computational Intelligence and Security, pp.
237-241, 2007.
its neighbouring objects, DEPTA fails to identify all records
[5] Gui-Sheng Yin, Guang-Dong Guo, and Jing-Jing Sun, “A
correctly. Template-based Method for Theme Information Extraction
from Web Pages,” IEEE International Conference on Computer
CONCLUSIONS Application and System Modelling (ICCASM 2010), vol. 3,
pp. 721-725, 2010.
One of the major issues in Web Content Mining is the
[6] Xunhua Liu, Hui Li, Dan Wu, Jiaqing Huang , Wei Wang, Li
extraction of precise and meticulous information from the Yu, and Ye Wu Hengjun Xie, “On Web Page Extraction based
Web. In this paper, we propose two similarity based mechan- on Position of DIV,” IEEE Computer and Automation
© 2013 ACEEE 48
DOI: 01.IJIT.3.1. 1102

Short Paper

Engineering (ICCAE), vol. 4, pp. 144-147, February 2010. Discovery,” Proceedings of the 2nd International Symposium
[7] Nilesh Dalvi, Phillip Bohannon, and Fei Sha, “Robust Web on Communications and Information Technology, pp. 97-114,
Extraction: An Approach Based on a Probabilistic Tree-Edit 2002.
Model,” Twenty-Eight ACM SIGMOD-SIGACT-SIGART [13] K. Bharat and M. R. Henzinger, “Improved Algorithms for
symposium on Principles of database systems, pp. 335-348, Topic Distillation in a Hyperlinked Environment,” Proceedings
2009. of the 21st Annual International ACM SIGIR Conference on
[8] Robert Novotny, Peter Vojtas, and Dusan Maruscak, Research and Development in Information Retrieval, pp. 104-
“Information Extraction from Web pages,” IEEE/WIC/ACM 111, 1998.
International Conference on Web Intelligence and Intelligent [14] C. Aggarwal, F. Al-Garawi, and P. Yu, “Intelligent Crawling on
Agent Technology – Workshops, pp. 121-124, 2009. the World Wide Web with Arbitrary Predicates,” Proceedings
[9] Tiezheng Nie, Zhenhua Wang, Yue Kou, and Rui Zhang, of the 10th International World Wide Web Conference, pp. 96-
“Crawling Result Pages for Data Extraction based on URL 105, May 2001.
Classification,” IEEE Computer Society, Seventh Web [15] S. Chakrabarti, K. Punera, and M. Subramanyam, “Accelerated
Information Systems and Applications Conference, pp. 79- Focused Crawling through Online Relevance Feedback,”
84, 2010. Proceedings of the 11th International Conference on WWW,
[10] Nikolaos K, Papadakis, Dimitrios Skoutas, and Konstantinos ACM, pp. 148-159, May 2002.
Raftopoulos, “STAVIES: A System for Information Extraction [16] M. Ehrig and A. Maedche, “Ontology-Focused Crawling of
from Unknown Web Data Sources through Automatic Web Web Documents,” Proceedings of the 2003 ACM Symposium
Wrapper Generation Using Clustering techniques”, IEEE on Applied Computing, pp. 1174-1178, 2003.
Trans. on Knowledge and Data Engineering, vol. 17, no. 12, [17] Srikantaiah K C, Suraj M, Venugopal K R, Iyengar S S, and L
pp. 1638-1652, December 2005. M Patnaik, “Similarity based Web Data Extraction and
[11] Chia-Hui Chang, Moheb Ramzy Girgis, “A Survey of Web Integration System for Web Content Mining,” Proceedings of
Information Extraction Systems,” IEEE Trans. on Knowledge the 3rd International Conference on Advances in communication,
and Data Engineering, vol. 18, no. 10, pp. 1411- 1428, October Network and Computing Technologies, CNC 2012, LNICST,
2006. pp. 269–274, 2012.
[12] N. Angkawattanawit, A. Rungsawang., “Learnable Crawling: [18] Bluetooth SIG Website, https://www.bluetooth.org/tpg/
An Efficient Approach to Topic-specific Web Resource listings.cfm

© 2013 ACEEE 49
DOI: 01.IJIT.3.1. 1102

Similarity based Dynamic Web Data Extraction and Integration System from Search Engine Result Pages for Web Content Mining

More Related Content

What's hot (18)

Similar to Similarity based Dynamic Web Data Extraction and Integration System from Search Engine Result Pages for Web Content Mining (20)

More from IDES Editor (20)

Recently uploaded (20)

Similarity based Dynamic Web Data Extraction and Integration System from Search Engine Result Pages for Web Content Mining