Re-using Media on the Web: Media fragment re-mixing and playout
Download as PPT, PDF0 likes2,784 views
The document discusses the re-mixing and playout of media fragments, focusing on the use of RDF (Resource Description Framework) for storing and querying data. It highlights the importance of RDF stores and the SPARQL query language to facilitate retrieval and manipulation of RDF data. Additionally, it covers the implementation of media fragment players and servers that support playback of media fragments, along with examples of relevant technologies and tools.
Re-using Media on the Web: Media fragment re-mixing and playout
- 1. 1 Media fragment re-mixing and playout Lyndon Nixon MODUL University Vienna [email protected] RE-USING MEDIA ON THE WEB WWW2014 Tutorial, Seoul, S Korea, April 8 2014
- 2. 06.01.14 Slide 2 of 32 3. Media fragment re-mixing and playout • Storing the data: repositories • Querying the data: retrieval • Re-mixing the media: semantics • Play out the media: clients 2
- 3. 06.01.14 Slide 3 of 32 Repositories
- 4. 06.01.14 Slide 4 of 32 Storing your RDF data Like any data, RDF needs to be stored. It can be serialised to different formats like XML and JSON, but this is not the same as the underlying data model. • RDF is a GRAPH, XML and JSON are TREEs -> the same RDF data can be serialised in different XML and JSON syntax • RDF datatypes have different semantics than XML and JSON datatypes In other words: if you want to persist and manipulate your data as RDF, you should use a RDF specific solution.
- 5. 06.01.14 Slide 5 of 32 RDF Stores A RDF Store (or RDF repository, or triple store) is a database specialized for RDF triples. •Can ingest RDF in different serialisations •Supports a RDF specific query language such as SPARQL •May support RDF insert/delete (a.k.a. SPARQL UPDATE) •Can support inferencing over RDF data Different implementations, can be broadly categorized into: •In-memory: Triples are stored in main memory (usually fastest) •Native: Persistence is provided by using a custom DB, where the database structure is optimised to the RDF model (most commonly used) •Non-native: Persistence is provided by using a third party relational DBMS (e.g. mySQL or postgres) (may be better if there are many updates or a need for good concurrency control)
- 6. 06.01.14 Slide 6 of 32 RDF Store Overview Good overview at http://www.w3.org/ 2001/sw/wiki/Categ ory:Triple_Store (47 listed stores) Scalability: http://www.w3.org/ wiki/LargeTripleSto res lists quoted large scale deployments of stores (up to 1+ trillion RDF triples) Comparison from 9/2012 at http://www.garshol.priv.no/blog/231.html
- 7. 06.01.14 Slide 7 of 32 OpenRDF Sesame Sesame is an open source RDF framework for RDF storage, query and inference. •http://www.openrdf.org •Implemented in Java •Supports in-memory, native and non-native stores. Features •Lightweight yet powerful Java API •SPARQL query language support (including SPARQL UPDATE) •Highly scalable •Can support inference via a RDF Schema reasoner •Transactionality •Context (a.k.a. Named Graphs) RDF Model Rio SAIL API SAIL Query Model SeRQL SPARQL Repository Access API HTTP Serverapplication application HTTP / SPARQL protocol
- 8. 06.01.14 Slide 8 of 32 Insert RDF into Sesame The central concept in Sesame is the repository. You add and query data by repository.
- 9. 06.01.14 Slide 9 of 32 Using context You can distinguish data sets in one repository by using context This is an additional URI attached to each triple you insert. (thus also referred to as quads) You can now query or update only on triples within one named context (in RDF, this is called a named graph) Context is useful for provenance tracking and for versioning If a query does not specify a context, the default context is used. It can be exclusive (only triples without a context) or inclusive (all triples in all contexts or none)
- 10. 06.01.14 Slide 10 of 32 Browse RDF in Sesame
- 11. 06.01.14 Slide 11 of 32 Retrieval
- 12. 06.01.14 Slide 12 of 32 SPARQL RDF query language RDF data is a „labeled, directed graph“ without any fixed vocabulary (this is defined in RDF Schema). A query language for RDF needs to support this data model: •Tranversing paths in a RDF graph •Being independent of any defined vocabulary •Able to query over the data or the schema SPARQL („sparkle“) is a W3C standard supported in most RDF tools •SELECT... WHERE.... constructions like in SQL •Path expressions •Additional keywords for more query expressiveness
- 13. 06.01.14 Slide 13 of 32 Examples: triple patterns The basic idea in SPARQL is to match graph patterns against RDF graphs.To understand graph patterns we must first define triple patterns: • A triple pattern is similar to a triple (in RDF) but with one or more variables in the place of a RDF resource (URI or literal) Triple: dbpedia:Lou_Reed foaf:givenName „Lewis Allen Reed“ . Triple pattern: dbpedia:Lou_Reed foaf:givenName ?name . ?name is the variable. If a RDF graph with the above triple is queried with the below triple pattern, then in the SPARQL results the variable ?name would be ‚bound‘ to the value „Lewis Allen Reed“. • A SPARQL query result is a set of bindings for the variables appearing in the SELECT clause, based on matching RDF resources to variables in triple patterns in the WHERE clause. .
- 14. 06.01.14 Slide 14 of 32 Examples: conjunctive and disjunctive query In graph patterns, several triple patterns are listed within braces { ... } and these are interpreted conjunctively: Ex: { ?what tech:noOfWheels „4“ . ?what tech:minSpeed „180“ . } The variable ?what will only be bound to resources which BOTH have 4 wheels AND a minimum speed of 180. You can also join results from distinct graph patterns using the UNION keyword. Note that result sets from graph patterns and from UNIONs are different, since UNION works disjunctively: Ex: { ?what tech:noOfWheels „4“ . } UNION { ?what tech:minSpeed „180“ . }
- 15. 06.01.14 Slide 15 of 32 Examples: FILTER, ORDER BY SPARQL has many other keywords. FILTER restricts variable bindings returned in query results to those for which the filter expression evaluates to true. A filter applies to solutions over the entire graph pattern it is contained in. Ex: { ?what tech:noOfWheels „4“ . ?what tech:minSpeed ?speed . FILTER ( ?speed > 170 ) } The ORDER BY keyword determines the sequence of query results returned according to a sort on the referenced variable‘s bindings, ascending by default: Ex: { ?what tech:minSpeed ?speed . } ORDER BY ?speed Or ORDER BY DESC(?speed)
- 16. 06.01.14 Slide 16 of 32 SPARQL in Sesame
- 17. 06.01.14 Slide 17 of 32 Query for media fragments: SPARQL-MM Research work in progress: extending SPARQL to Media Fragments by adding spatio-temporal filter and aggregration functions. Relation Function Aggregation Function Spatial mm:rightBeside mm:spatialIntersection mm:spatialOverlaps mm:spatialBoundingBox … … Temporal mm:after mm:temporalIntersection mm:temoralOverlaps mm:temporalIntermediate … … Combined mm:overlaps mm:boundingBox mm:contains mm:intersection Courtesy Thomas Kurz (Salzburg Research) & MICO EU project http://demos.mico-project.eu/sparql-mm/sparql-mm/demo/index.html
- 18. 06.01.14 Slide 18 of 32 Semantics
- 19. 06.01.14 Slide 19 of 32 Semantic search? Broadly, any search paradigm which makes use of machine understanding of the data being queried. Search term „Presidents of the United States“ CLASSICAL SEARCH Return all media with the string „Presidents of the United States“ in their title or description SEMANTIC SEARCH Return all media with an occurance of an instance of the type „Presidents of the United States“ in their metadata, e.g. George W Bush, Bill Clinton, Barack Obama ...
- 20. 06.01.14 Slide 20 of 32 Search by types or categories The set of „Presidents of the United States“ The category of „Presidents of the United States“ World Leaders Presidents Presidents of the United States Bill Clinton Barack Obama George W Bush Barack Obama George W Bush Bill Clinton
- 21. 06.01.14 Slide 21 of 32 Search by paths in the graph RDF can be more expressive through defining properties with specific semantics; semantic search can be more powerful by using these semantics However, independent of a RDF vocabulary, shared graph properties can be an indicator of resource similarity & used in domain-independent semantic search ? ? < 50 England Age BornIn ?v ? ? ?p ?p
- 22. 06.01.14 Slide 22 of 32 Search with Linked Data Linked Data: RDF metadata published on the Web, with information about each resource discoverable via its unique URI. Machines can transverse the RDF graph by looking up each URI in turn. DBPedia: an extraction of the knowledge in Wikipedia (esp. the info boxes) as RDF and published in its own namespace (http://dbpedia.org) Each Wikipedia article of the form http://en.wikipedia.org/wiki/X has a DBPedia resource at the URI http://dbpedia.org/resource/X Has its own ontology which defines resource properties and classes, also maps the Wikipedia categories using the dcterms:subject property. Advantages: Linked Data URIs allow an unambiguous reference to information about a concept, that information carries the authority of the data provider, all applications have the same understanding of the concept Disadvantages: information at the URI is not under your control, sometimes messy/dirty (the case with DBPedia) use local copies, extend with own metadata
- 23. 06.01.14 Slide 23 of 32 SPARQL endpoints and the SERVICE keyword Linked Data sources usually provide a SPARQL endpoint for their datasets • A SPARQL endpoint is a SPARQL query processing service that supports the SPARQL protocol (http://w3.org/TR/rdf-sparql-protocol): a definition of how to send a SPARQL query to a Web Service and receive a query result. e.g. DBPedia.org has a SPARQL endpoint at http://dbpedia.org/sparql See a fuller list at http://esw.w3.org/topic/SparqlEndpoints SPARQL 1.1 includes the SERVICE keyword to include queries to SPARQL endpoints within a SPARQL query addressing a different repository. • With this in Sesame, you can SPARQL query a local repository BUT include bindings from external datasets within the query Ex: SERVICE <http://dbpedia.org/sparql> { ?resource dcterms:subject . ?subject . ?related dcterms:subject ?subject . }
- 24. 06.01.14 Slide 24 of 32 SPARQL for semantic search
- 25. 06.01.14 Slide 25 of 32 Clients
- 26. 06.01.14 Slide 26 of 32 What is a Media Fragment Player? Regular media players playback full media assets, only the consumer may seek within the media. A media fragment player supports playback of (spatial and/or temporal) media fragments in place of the full media asset, the customer doesn‘t have to do anything. We mean a player which can correctly interpret a media fragment as per the W3C Media Fragment URI specification.
- 27. 06.01.14 Slide 27 of 32 Media Fragments – Web browser support • Mozilla Firefox, since version 9 • Apple Safari, since Jan 2012 • Google Chrome, since Jan 2012 Focus on support for the temporal dimension in the HTML5 video player.
- 28. 06.01.14 Slide 28 of 32 Media Fragments – Javascript libraries • mediafragment.js: https://github.com/tomayac/Media-Fragments-URI • xywh.js: https://github.com/tomayac/xywh.js Focus on implementing the spatial and clock dimensions within browsers.
- 29. 06.01.14 Slide 29 of 32 Media Fragments – SyNote player http://smfplayer.synote.org/smfplayer/ • Cross-browser/device solution for Media Fragment playback via JavaScript • Supports Media Fragment playback from online video platforms such as YouTube and DailyMotion • Wider video format support than native HTML5 (e.g. also Flash or Windows Media)
- 30. 06.01.14 Slide 30 of 32 What about Media Fragment servers? The advantage of media fragment playback should be also that ONLY the content of the media fragment needs to be provided by the server to the client. (in reality, Web servers are still serving the whole media asset, the fragment is determined on the client) Implementing Media Fragment support on the server side requires new implementations of Web (media) servers... Ninsuma Media Server is a prototypical demonstration of how a server should function with Media Fragments. http://ninsuna.elis.ugent.be/MediaFragmentsServer
- 31. 06.01.14 Slide 31 of 32 Our Media Fragment Jukebox http://www.mediamixer.eu/jukebox/ox/
- 32. 32 Wrap up