Ad
More Related Content
What's hot (19)
Viewers also liked (20)
Ad
Similar to Semantic Search Over The Web (20)
Ad
Recently uploaded (20)
Semantic Search Over The Web
- 2. Introduction • Semantic search is to improve the accuracy of the search process by understanding the context and limiting the ambiguity, • Semantic search is to make the semantics of the Web content machine understandable. • The semantic Web creates associations between different representations of the same real-world entity. • They allow data from many different sources to be interlinked (linked open data cloud). • Existing solutions are either search engines that simply index the semantic Web data or the traditional search engines enhanced with some basic form of synonym usage, as supported by Google and Bing. • The semantic Web is a huge distributed database we can query to get information coming from different sources.
- 4. Resource Description Framework (RDF) • All data items in RDF are uniformly represented as triples of the form (subject, predicate, object) or (subject, property, value) triples. • RDF extends the linking structure of the Web to use URIs to name the relationship between things as well as the two ends of the link. • This linking structure forms a directed, labeled graph. • The graph view is the easiest possible mental model for RDF.
- 5. Advantages of RDF • RDF offers a standardized and flexible framework for publishing structured data on the Web such that • (1) data can be linked, incorporated, extended, and reused by other RDF data across the Web; • (2) heterogeneous data from independent sources can be automatically integrated by software agents; • (3) the meaning of data can be well defined using ontologies
- 6. Web of Data • Today, most Web sites are generated from structured data that is stored in relational databases. • The main benefit of using the ontology is that the corresponding data are clean and well structured. • A lot of Web sites that embed structured data into HTML pages. • Google, Yahoo!, and Microsoft have jointly agreed on a set of vocabularies for describing over 200 different types of entities. • Question: • “How can we embed structured data into HTML pages and link them each other?”
- 7. Topology of the Web of Data • Microformats • Microformats is a technique for marking up structured data about specific types on entities. • RDFa • W3C started in 2004 to standardize RDFa as an alternative. • Microdata • Microdata is an alternative proposal for embedding structured data into Web pages which was initially presented as part of the HTML 5 standardization effort in 2009. • Linked Data • The term Linked Data refers to a set of best practices for publishing structured data directly on the Web.
- 8. Microformats • Designed for humans first, machines second. • Microformats requires the development of specialized parsers for each format. • Microformats is used to address specific use cases. • Microformats consist of a definition of a vocabulary (names for classes and properties), as well as a set of rules (e.g., required properties, correct nesting of elements). • HTML/XHTML attributes are used for inserting markup. • The microformats community encourages mixing microformats and reusing existing formats when creating new ones.
- 9. Microformats Syntax • Figure shows Microformat representation of the example data Peter Smith. • The vcard is a root class name indicating the presence of an hCard. • The properties are url (Peter’s home page) and fn (full name). • The markup also states that Peter knows Paula with the property met acquaintence.
- 10. Microformats Deployment on the Web • Yahoo! Search are indexing semantic markup including hCard, hCalendar, hReview, hAtom, and XFN. • Google are parsing the hCard, hReview, and hProduct microformats and using them to populate search result pages. • Facebook publishes event pages annotated with hCalendar, • Yelp.com adds hReview and hCard to all of their listings • Wikipedia templates are able to automatically generate microformats such as geo, hCard, and hCalendar markup.
- 11. RDFa syntax • RDFa allows one to embed RDF triples within the HTML document object model (DOM). • The RDFa syntax specifies how HTML elements may be annotated with entity identifiers, entity types, string properties, and relationship properties. • The HTML attribute @about indicates that the entity identified by the URI reference http://example.com/Peter. • The HTML attribute @rel specifies a relationship property between the HTML element and the target URL. • The property foaf:knows to state that Peter knows Paula. • For string properties, the attribute @property (foaf:name) to express Peter’s name. • A central idea of RDFa is the support for multiple, decentralized, independent, extensible vocabularies, in contrast to the community-driven centralized management of microformats.
- 12. Microdata • Microdata is an attempt to provide a simpler alternative to RDFa and Microformats. • It defines five new HTML attributes (as compared to zero for Microformats and eight for RDFa), • Provides a unified syntax (in contrast to Microformats), • Allows for the usage of any vocabularies (similarly to RDFa). • W3C currently has two draft specifications (Microdata and RDFa) with the same objective.
- 13. Microdata Syntax • Microdata consists of a group of name–value pairs. • The groups are called items, and each name–value pair is a property. • In order to mark up an item, the itemscope attribute is applied to an HTML element . • To add a property to an item, the itemprop attribute is used.
- 14. Linked Data • The term Linked Data refers to a set of best practices for publishing structured data directly on the Web. • Linked Data uses hyperlinks to connect disparate data into a single global dataspace. • A Linked Data application that has looked up a URI and retrieved RDF data by following links. • In a Linked Data context, if an RDF link connects URIs in different namespaces, it ultimately connects resources in different datasets.
- 15. Linked Data Principles 1. Use HTTP URIs as names for things. 2. When someone looks up a URI, provide useful information, using recommended standards (RDF, SPARQL). 3. Include links to other URIs, so that they can discover more things. 4. Whenever a Linked Data client looks up an HTTP URI over the HTTP protocol, the corresponding Web server returns an RDF description of the identified object using the RDF/XML syntax.
- 16. Linked Data(RDF/XML) Syntax • FOAF, a vocabulary for describing people. • URI http://example.com/Peter of type foaf:Person. • foaf:name states that this thing has the name Peter Smith. • Foaf:knows states that Peter Smith knows Paula Jones, which is identified by the URI reference http://example.com/People/Paula.
- 17. Evaluation Data For Search Engines • A number of publicly available evaluation datasets that have been crawled from the Web and can be used for evaluating semantic search applications: • ClueWeb09 • TREC Entity • CommonCrawl • WebDataCommons • Sindice • Billion Triple Challenge • SemSearch • Or to obtain Web data, use publicly available software for crawling the Web, such as Nutchfor crawling Web pages and LDSpider for crawling Linked Data
- 18. Challenges of “Web of Data” • Applications that want to exploit the Web of Data are facing two main challenges today: • Semantic Heterogeneity. The different techniques that are used to publish data on the Web lead to a certain degree of syntax heterogeneity • Data Quality. The Web is an open medium and everybody can publish data on the Web. Thus, the Web will always contain data that is outdated, conflicting, or intentionally wrong (spam).
- 20. Perspectives to storage and indexing of RDF datasets • The Relational Perspective • An RDF graph is just a particular type of relational data, and that techniques developed for storing, indexing, and answering queries on relational data. • The Entity Perspective • Resources in the RDF graph are interpreted as “objects” or “entities”. Each entity is determined by a set of attribute–value pairs in the entity perspective. • The Graph-Based Perspective • The focus is on supporting navigation in the RDF graph when viewed as a classical graph in which subjects and objects form the nodes, and triples specify directed, labeled edges.
- 21. Storing and Indexing Under the Relational Perspective • Two different approaches for storing RDF data in relational databases. • The vertical representation: • Stores all triples in an RDF graph as a single table over the relation schema (subject, predicate, object). • Due to the large size of the RDF graphs and the potentially large number of self-joins required to answer queries. • The horizontal representation approach interprets triple predicate values as column names, and stores RDF graphs in one or more wide tables.
- 23. Disadvantages and Advantages • There is a weakness when answering queries that do not specify the predicate value. • The relational schema must be changed whenever a new predicate value is added to the RDF graph. • On the positive side, the horizontal representation makes it easy to support typing of object values. • it is easy to integrate existing relational data with RDF data.
- 24. Storing and Indexing Under the Entity Perspective • Resources in the RDF graph are interpreted as “objects,” or “entities.” • Each entity is determined by a set of attribute–value pairs. • Heavy use of the inverted index data structure. • Typically, the following two general types of queries are to be supported • Simple keyword queries: A keyword query returns all entities that contain an attribute, relationship, and/or value relevant to a given keyword. • Conditional entity-centric queries: A conditional entity-centric query returns all known entities that satisfy some given conditions on a combination of attribute, relationships, and values at the same time
- 25. Storing and Indexing Under the Graph-Based Perspective • The focus is on supporting navigation in the RDF graph in which subjects and objects form the nodes, and predicates specify directed, labeled edges. • Typical query patterns are graph-theoretic queries such as reachability between nodes. • The major issue under this perspective is how to explicitly and efficiently store and index the implicit graph structure. • A structural index is used to obtain a reduced version of this graph where certain nodes have been merged while maintaining all edges.
- 26. Further Index Researches • A major open issue is the incorporation of schema and ontology reasoning (e.g., RDFS and OWL) in storage and indexing. • A little work on the impact of reasoning on disk-based data structures. • Efficient maintenance of storage and indexing structures as datasets. • In the entity perspective, investigation of support for richer query languages and integration with techniques from the other two perspectives. • Study of richer structural indexing techniques and related query processing strategies.
- 27. Semantic Wiki • Semantic wikis are wikis that add machine-processable annotations to wiki pages. • Annotations exists for data items, most frequently wiki pages and tags, but also smaller portions of text. • The annotations may be freely chosen tags, or more formal mechanisms such as RDF backed by (imported) RDFS or OWL ontologies are offered as well. • The annotations may be used for some processes: consistency checking, improved navigation, search, querying, personalization, context-dependent presentation, and reasoning.
- 28. Semantic Wiki Queries • Annotations are often represented in RDF. They are compatible with SPARQL. • Semantic wikis usually provide simple full-text search for the querying of textual content or RDF literals. • A standard RDF query language such as SPARQL or RDQL can often be used for querying the annotations. • A number of semantic wikis also come with their own language for querying annotations(i.e.,Kiwi-KWQL).
- 29. DBpedia • DBpedia is extracted structured content from Wikipedia. • This structured information is made available on the World Wide Web. • DBpedia allows users to semantically query relationships and properties of Wikipedia resources • DBpedia is including links to other related datasets. • It is possible to ask complex queries to the Dbpedia with SPARQL endpoint.
- 32. Keyword-based search systems address the following key steps: • Composing a valid semantic query, • for a user it is difficult to master a query language (e.g., SPARQL) and acquire sufficient knowledge about the ontology or the schema of the data source. • Identifying (substructures holding) data matching input keywords, • by using an indexing system or a database engine. Indexing may be made by shortest path to root nodes. • Linking identified data (substructures) into solutions • since data is usually scattered across multiple places, e.g., in different tables or XML elements. • Ranking solutions according to a relevant criterion (i.e., a suitable scoring function). • Specific implementation of TF/IDF may be used for scoring keyword elements. • Only the top-k solutions with highest score, are returned to the users as query answers.
- 33. The interface of Interactive Query Construction of QUICK consists of three parts: • Quick creates Semantic Queries from keywords: • A search field (on the top), • The construction pane showing query construction options (on the left), • The query pane showing semantic queries (on the right).
- 34. Performance Measurements of Search • Three measures have been proposed to evaluate performance: • Exhaustivity measures the relevance of a solution in terms of the number of keywords it contains. • Specificity measures the precision of a solution in terms of the number of keywords it contains with respect to other irrelevant terms occurring in the solution. • Overlap measures the information content of a solution in terms of its intersection with other solutions. • Clearly, the best ranking strategy balances exhaustivity and specificity while reducing overlap.
- 35. Semantic Web Search Engines • Hidden Web/Deep Web Approaches • RDF-Centric Search Engines • Distributed Web Search Architectures
- 36. Hidden Web/Deep Web Approaches • A vast amount of the information available on the Web is hidden behind sites with heavy dynamic content, usually backed by relational databases. • Manually constructed, site-specific wrappers to extract structured data from HTML pages or to communicate directly with the underlying database of such sites. • Automaticall crawler exists, however, this approach is “task specific” and not appropriate for general crawling. • The Semantic Web may represent a future direction for bringing Deep Web information to the surface by using RDF as a common and flexible data model.
- 37. RDF-Centric Search Engines • Early prototypes are Ontobroker and SHOE using the concepts of ontologies and semantics on the Web. • Swoogle offers search over RDF documents by means of an inverted keyword index and a relational database. • Watson also provides keyword search facilities over Semantic Web documents but additionally provides search over entities. • Sindice is a registry and lookup service for RDF files based on Lucene and a MapReduce framework. • Falcons search engine offers entity-centric searching for entities (and concepts) over RDF data. • It has rank entities by using a logarithm of the count of documents in which they are mentioned. • GoWeb system demonstrates the benefit of searching structured data for the biomedical domain.
- 38. Distributed Web Search Architectures • Distributed architectures have long been common in traditional Web search engines. • The system architectures includes an incremental crawler, ranker and storage manager, indexer, and query processor. • Some systems use a distributed inverted index • (based on an embedded database system) over a large corpus of Web pages, for subsequent analysis and query processing.
- 40. SWSE • SWSE consists of crawling, data enhancing, indexing and a user interface for search, browsing and retrieval of information; operates over RDF Web data (Linked Data). • SWSE allows users to specify keyword queries in an input box and responds with a ranked list of result snippets. • The results refer to entities not documents (entity search over instance data). • Users can subsequently navigate to related entities, as such, browsing the Web of Data.
- 41. SWSE Preprocessing • The crawler accepts a set of seed URIs and retrieves a large set of RDF data from the Web, • The consolidation component tries to find synonymous (i.e., equivalent) identifiers in the data, and combines the data according to the equivalences found, • The ranking component performs links-based analysis over the crawled data and derives scores indicating the importance of individual elements in the data (PageRank). • The reasoning component produces new data which is implied by the inherent semantics of the input data, • The indexing component prepares an index which supports the information retrieval tasks required by the user interface (Inverted Index).
- 42. SWSE Query Processing • With the distributed index built and prepared on the slave machines, the query processor is able to accept user queries. • For a top-k keyword query, the coordinating machine requests k result identifiers and ranks from each of the slave machines. • The coordinating machine then computes the aggregated top-k hits. • To provide the raw data required, the master machine directly requests data from the respective slave machine (focus view).
- 45. A Recommender System for Linked Data MORE (MORE than Movie Recommendation) • The systems is needed to recommend items based on user preferences. • The systems should allow an easy and friendly exploration of the information/data related to a particular domain of interest. • New challenges with the huge amount of interlinked data coming from the semantic Web.
- 46. Semantic Vector Space Model (MORE) • In VSM, weights are assigned to index terms in queries and in documents (sets of terms), • Weights are used to compute the degree of similarity between each document in the collection and the query. • Whole RDF graph may be represented as a three-dimensional tensor where each two-dimensional slice refers to an ontology property. • Given a property, each movie is seen as a vector, whose components refer to the TF-IDF (resource frequency-inverse movie frequency). • For a particular property, the similarity degree between two movies is represented by the correlation between the two vectors. • To obtain the global correlation between two movies, a weighted sum of each property is calculated.
- 50. Exploratory Search Applications • They are designed to satisfy the needs of users with specific aims. • They supports the publishing and integration of data sources for vertical domains. • The user will be able to select sources based on individual or collective trust. • And systems will be able to route queries to such sources and to provide easy to-use interfaces for combining them within search strategies.
- 51. Deployment Architecture • The deployment of exploratory Web applications integrating data sources requires a number of software components and sophisticated interactions between them: • The processing modules in charge of invoking services that query the data sources. • The execution engine is a data and control-driven query engine specifically designed to handle multidomain queries. • The control layer is the controller of the architecture; it is designed to handle several system interactions. • The repository contains the set of components and data storages used by the system.
- 52. Exploratory Search Applications Examples • Night Planner • Weekend Browser • Real-Estate Browser • Job-House Combination Browser
- 55. Real-Estate Browser • A real-estate browser is a long-lived, hierarchical application. • It is centered around a real-estate. • A user may select some house offers and evaluate them according to some search dimensions (e.g., distance from work, school). • The designer may simplify the interaction by combining several services into one query (e.g., walkability and vicinity to markets and parks)
- 57. References • Semantic Search Over The Web, Roberto De Virgilio, Francesco Guerra, Yannis Velegrakis, Springer, 2012. • https://rdfa.info/ • http://microformats.org/wiki/Main_Page • https://schema.org/docs/gs.html • http://wiki.dbpedia.org/