Lecture 8 – Documentation,

Hypertext and MHEG

 Document, Architecture and Multimedia
integration;
 Hypertext, Hypermedia and Multimedia
 Hypermedia System: Architecture, nodes
and Pointers;
 Document Architecture: SGML and ODA
 MHEG

1
Introduction
 A Document consists of a set of structural information that can
be in different forms of media
 During presentation it can be generated and recorded.
 A document is aimed at the perception of a human and is
accessible for computer processing
 A multimedia document is a document which is comprised of
information coded in at least one continuous (time dependent)
medium and in one discrete (time independent) medium.
 Integration of the different media is given through a close
relation between information units.
 This is also called synchronization.
 It is closely related to its environment of tools, data
abstractions, basic concepts and document architecture.

2
Documents
 Currently continuous and discrete data are processed differently:
 Text is processed within an editor program as a type of a programming
language (namely the type character)
 A motion picture can be manipulated with the same editor program only
through library calls.
 The goal of abstracting multimedia data is to achieve integration all
media.
 This reduce the complexity of the program generation and
maintenance that process multimedia data.
 Basic system concepts for document processing use multimedia
abstractions and also serve as concepts for the information
architecture in a documents
 Thus we use the terms document architecture and information
architecture interchangeably.

3
Document Architecture
 Exchanging documents entails exchanging the document
content as well as the document structure.
 This requires that both documents have the same
document architectures.
 The current standardized architectures are :
 The Standard Generalized Markup Language (SGML) and
 The Open Document Architecture (ODA).
 There are also proprietary document architectures such
as
 DEC’s Document Content Architecture (DCA)
 IBM’s Mixed Object Document Content Architecture (MO:DCA)
 Information architectures use their data abstractions and
concepts.
 A document architecture describes the connections
among the individual elements represented as models
4
 Elements in Document
Architecture
 The manipulation
model describes all
Presentation the operations
Structure allowed for creation,
Model change and deletion
Manipulation of multimedia
information.
Model  The representation
Content model defines
 Protocols for
exchanging
information between
computers
 Formats for storing
data
Representation  Presentation model
Model describes the format
for the presentation
5
Manipulation of Data
 The user becomes most aware of multimedia documents
through tools for manipulation of multimedia data, such
as editors, desktop publishing programs and other text
processing programs
 A document undergoes the process shown in figure next.
 The information included in a document belongs to a
certain document type.
 Eg. Business letter
 Internal Memorandum
 The same document can belong to other types which
mainly influence the final representation.
 The transformation from the actual information to its
final representation behaves according to rules specified
6
Manipulation of Data
t
Documen  The user becomes most
aware of multimedia
documents through tools
Source for manipulation of
Source
Document multimedia data.
Type  Such as editors, desktop
publishing programs and
other text processing
Rules Processing programs
 The document undergoes
Target the process shown aside.
Document  The information included
Type in a document belongs to
a certain document type.
t
Documen
Eg. Business letter or
Target
internal memorandum
7
Manipulation of Multimedia Data
 The same document can belong to other types which mainly
influence the final representation.
 The transformation from the actual information to its final
representation behaves according to rules specified to the
document architecture.
 The processing cycles of a traditional document and an
interactive multimedia presentation are analogous.
 The document exists in a processable representation.
 The result is a final representation of the document.
 A typical example of this representation is the typesetting
language PostScriptTM.
 The availability of hypertext and multimedia technology have
changed the representation of documents although the
processing cycle remains same.

8
Manipulation of Multimedia Data
 The output of interactive hypermedia documents will be mostly
computer-supported.
 Therefore, the presentation of a documents will have to be not
only final, but also be executable.
 While there are a broad range of processable formats, there are
too few final representation formats.
 It has been internationally recognized that such a final
representation is very important and, especially in a distributed,
heterogeneous system environment.
 This exchange format for interactive multimedia presentation is
called MHEG (Multimedia and Hypermedia Information Coding
Expert Group)
 Using the main concept of Hypermedia and Hypertext for
multimedia documents, SGML, ODA are presented.

9
Hypertext, Hypermedia and
Multimedia
 Communication reproduces knowledge stored in the
human brain via several media.
 Documents are one method of transmitting information.
 Reading a document is an act of reconstructing
knowledge.
 In an ideal case, knowledge transmission starts with an
author and ends with a reconstruction of the same ideas
by a reader.
 Information loss is minimal.
 Knowledge must be artificially serialized before the
actual exchange.
 Hence it is transformed into a linear document and the
structural information is integrated into the actual
content.
10
Hypertext and Hypermedia
 In the case of hypertext and hypermedia, a graphical
structure is possible in a document which may simplify the
writing and reading process.
 A book or an article on a paper has a given structure and is
represented in a sequential form. Although it is possible to
read individual paragraphs without reading previous
paragraphs, authors mostly assume a sequential reading.
 Therefore many paragraphs refer to previous learning in
the document.
 Novels, movies always assume a pure sequential reception.
 Scientific literature can consists of independent chapters,
although mostly a sequential reading is assumed.
 Technical documentation consists often a collection of
relatively independent information units.
11
Properties of Hypertext and
Hypermedia
 Non Linear Information Chain
 They have as a major property a non linear information link
 There exists not only a reading sequence, but also the reader
decides on his/her reading path.
 The reader can start in a lexicon with a notion hypertext, then go
through a cross reference to systems and finish with a description of
AppleTalk.
 By this association, through reference links, the author of the
information determines the actual links
 The structure is a tree where the reading path in this linear
document is explained verbally and not through the structure.
 A hypertext structure is a graph, consisting of nodes and edges.
 The references to other chapters and literature citations are such
pointers which build a tree-similar document to a graph.
12
Non Linear Information Chain cont…
 The nodes are the actual information units.
 They are the text elements, individual graphics, audio or video
LDUs.
 The information units are shown at the user interface mostly in
their own windows.
 The edges provide links to other information units.
 They are usually called pointers or links.
 A pointer is mostly a directed edge an includes its own
information too.

13
Anchor
 The forward movement in linear sorted documents is
called a navigation through the graph.
 At the user interface, the origin of pointer must be
marked, so that the user can move to a further
information unit.
 This origin of a pointer is called an anchor
 A main factor of the user interface is the concept of
the anchor.

14
How can the anchor be represented
properly
 A media-independent representation can happen through the selection
of general graphical elements, such as buttons.
 In such an element, information about the destination node should be
included.
 If the destination node is a text, a short, descriptive text of the content
can be represented.
 In the case of an image, the image content can appear in minimized
form on the screen.
 A visual representation of the video content can follow in form of a
moving icon (MICON).
 This is a minimized motion picture which represents a characteristical
portion of the video sequence of the destination node.
 If the content of the destination node consists of audio information, a
visual representation of the audio content must follow.

15
Anchor cont…
 In a text, individual words, paragraphs or text sections of
different length can be used for representation.
 The positioning of the pointer to the marked area and double
clicking in this area leads to a display of the destination node,
connected with the clicked information.
 In images, specific graphical objects or simply areas are defined
as selection objects, a specific marking can occur through a color
or stripe.
 In a motion video, media-independent representation of the
anchor are preferred.
 There can also be time changing areas used.
 Mostly, no spatial selection occurs and the particular shown
image is conclusive.
 A time selection is supported.
16
Hypertext system
 This system is mainly determined though non-linear
links of information.
 Pointers connect the nodes.
 The data of different nodes can be represented with
one or several media types.
 In a pure text system, only text parts are connected.
 Hypertext as an information object which includes
links to several media.
 Multimedia System
 It contains information which is coded at least in a
continuous and discrete medium

17
Hypermedia System
 Hypermedia System
 Includes the non linear information links of hyper text
systems and the continuous and discrete media of
multimedia systems.

Multimedia Multimedia Hyper Text

 In often the case, hypermedia notion is not use in its

strongest sense.
18
 Hypertext and hypermedia are used interchangeably.
Hypermedia System
 A hypermedia system includes the non-linear
information links of hypertext systems and the
continuous and discrete media of multimedia
systems.
 There have been many international conferences
covering this area since the late 1980s:
Hypertext’87, Hypertext ’89 etc.
 There exists a large number of conferences and
workshops, in addition to these main international
events, at the regional and national levels.

19
Hypermedia Systems: History
 The history of hypertext goes quite far back, although it
has been only recently that hypertext systems came on
the market.
 Vannever Bush is the originator of the main hypertext
concept, the linked information structure.
 He described the first hypertext system MEMEX (Memory
Extender).
 It was never implemented and remained in the paper
only.
 He developed this idea in 1932.
 He published the first descriptive article as We May
Think in 1945.

20
Hypermedia Systems- History
 Doug Englebart developed a project to augment
the human capability Augument at the Stanford
Research Institute (SRI) 1962-1976.
 One part of it is NLS(oN Line System), which has
hypertext properties.
 NLS served as joint document storage for all
created documents during this project.
 All scientists working on this project used it with
its possibilities of pointers.
 At the end there were approximately 100,000
entries.

21
Hypertext System : History
 Ted Nelson used the notion Hypertext for the first time in
1965.
 In his system all information which human beings
described at any time, was contained.
 His concepts described the access to local, as well as to
remote data.
 It was not implemented with his global information
content until now.
 Since the middle of 1960s, work on hypertext systems has
been going on at Brown University, Providence, RI.
 In 1967, the Hypertext Editing System was developed
under the leadership of Andries van Dam.
 This was the first run-able hypertext system.
 It needed 120 Kbyte main memory of a small IBM/360.
22
Hypermedia System - History
 It was sold and used for the documentation of
the Apollo Mission.
 The successor project was FRESS (File Retrieval
and Editing SyStem) in 1968.
 Both systems linked documents through
pointers, the user interface was implemented
through text.
 At Brown University from this time, successful
research in the area of hypertext/hypermedia
has continued.

23
 Hypermedia System - History
 The Aspen Movie Map is the first important hypermedia
system which supports continuous media.
 It was developed at the MIT Architecture Machine Group
under the intensive cooperation of Andrew Lippman.
 This group was built up later on with other scientists as was
known as the MIT Media Lab.
 With this application, a virtual drive through the city Aspen
(Colorado) could be followed on the computer screen.
 The user could move in all four geographical directions as
he/she desired.
 A joystick served as an input of the direction.
 The technique uses a large set of individual images which
were stored on a video stick.
24
Multimedia System - Concept
 Hypertext systems differ from each other in their
fundamental concepts
 Unspecified systems were not developed for any
specific application. They are determined to be used
generally for the generation and reading of hypertext
documents.
 Application specific systems were developed for
determined usages.

25
 Systems: Architecture, Nodes
and Pointers
 Architecture:
 The architecture of a hypertext system can be divided
into three layers with different functionalities
 Presentation Layer
 At the upper layer all functions connected to the user interface are
embedded.
 Nodes and pointers are mapped to the user interface.
 At the user interface, one or several parts of the document are
visualized.
 This layer determines which data are presented and how they are
presented.
 This layer takes over control of all inputs.

26
Systems: Architecture, Nodes
and Pointers
 Hypertext Abstract Machine
 The hypertext abstract machine (HAM) is placed between the
presentation and storage layers.
 It can expect from the underlying layer database functions
for storage of multimedia data in a distributed environment.
 It does not have to consider input and output of the upper
layer.
 HAM knows the structure of the document, it has the
knowledge about the pointers and its attributes.
 The data structure is constructed for the management of the
document.
 This layer has the least system dependency in comparison to
the other two layers
 This is the most suitable layer for standardization.
27
Systems: Architecture, Nodes
and Pointers
 Storage Layer
 The storage layer is the lowest layer.
 All functions connected with the storage of data belongs to
this layer.
 The specific properties of the different discrete and
continuous media need to be considered.
 Functionalities from traditional database systems are
expected
 Persistence – data persists through programs and processes
 Multiuser operations – Synchronizations, locks etc
 Restoration of data after failure – Transactions
 The nodes and pointers of a hypertext document are
processed as data objects without any special semantics.

28
Systems: Architecture, Nodes
and Pointers
 Unfortunately, in most current implementation,
there is no clear division between the different
layers.
 The reasons are
 Shorter development time
 Efficient implementation
 Currently an incomplete

29
Nodes
 A node is an information unit (LDU) in a hypertext
document.
 The main classification criterion of different realizations
is the maximal stored data amount in one node
 The maximal stored data amount can be limited and mapped
onto the screen size.
 The metaphor of note card, frame is introduced.
 A video clip and audio passage could be limited to the duration
of 20 sec (eg)
 An author is forced eventually to distribute logical connected text
content to several cards, although not desired.
 Applying it to the video clips it could link to a close
interconnection among the distributed sequences could get lost
easily.

30
Nodes
 Window based systems with an unlimited data amount
per node are the alternatives.
 Forward and backward scrolling of pages is offered
analogous to other windows at the user interface.
 Intermedia is such a system.
 Here at every node the amount of data, coded as
continuous media, is not limited with respect to its
duration.
 Therefore, individual nodes can include a very different
length although they appear to be equal
 For this problem two solution could be put forwards
 Either it is switched between the nodes
 Scrolling is used in one node with the usual mechanisms known
window systems.
31
Pointers
 Pointers are the edges of a hypertext graph.
 Hypertext systems are classified according to different
criteria with respect to edges.
 Which information includes a pointer?
 Simple pointers link two nodes of the graph without containing
any further information.
 They are visible only through the relation between the nodes
 Typed Pointers includes further information
 Each pointer gets label.
 Though this label, commentaries to the particular label are
possible.
 One can use further semantics.

32
Pointers
 The following relations can be expressed though
pointers:
 To Be: Relation
 To present :Demonstration
 To influence : Consequences from a behavior can be
described more closely.
 To need or to be needed : This relation expresses a
necessity
 To own : Ownership is expressed.
 To include : An inclusion relation is expressed in
different meanings.
 To be similar : Similarities can be explained.

33
Pointers
 Who is responsible for the pointer?
 Implicit Pointers : A relation between nodes can be
established automatically by a hypertext system.
 The author determines the algorithm according to which
pointers are created.
 The system Intermedia automatically generates all pointers.
 Query references are done automatically using main notions
of an entry.
 Explicit Pointers : The author creates all links
 A pointer can be created at different times.

34
Tools
 A hypertext system consists of several necessary tools.
 Editors process information represented in different
media.
 The generation, management, editing and deletion of
pointers are supported.
 Search tools allow the search of desired information.
 Browser allows a shortened but clear representation of
the nodes and edges.
 The nodes are described media dependently
 During navigation through a document, a proper support
of the phenomena is needed.

35
SGML (Standard Generalized
Markup Language) Architecture
 The SGML was supported mostly by American publishers.
 Authors prepare the text in a uniform way the title,
tables, etc without the description of the actual
representation
 The publishers specifies the resulting layout.
 The basic idea is that the author uses tags for marking
certain text parts.
 SGML determines the form of tags but it doesnot specify
their locations and meanings.
 User groups agree on the meaning of the tags.
 SGML makes a frame available with which the user
specifies the syntax description in an object specific
system.
36
SGML (Standard Generalized
Markup Language) Architecture
 Here classes and objects, hierarchies of classes and
objects, inheritance and the link to methods can be used
by the specifications.
 SGML specifies the syntax but not the semantics

<title>Multimedia – Technology </titile>

<author>Felix Getou </author>
<side> IBM </side>
<summary> This exceptional paper from
peter…………………</summary>

 This example shows an application of SGML in a text

document

37
 Figure : SGML:
Document processing
— from the information
to the presentation.

38
 Figure : SGML: Document
architecture — emphasis
on the representation
model.

39
Documentation Architecture ODA
 Open Document Architecture (ODA) initially
called office Document architecture as it
supports mostly office oriented application.
 Main goal is to support the exchange processing
an presentation of documents in open system.
 Used mainly by computer industry especially in
Europe.

40
ODA cont…

 Figure : ODA: Content, layout and logical view.

41
Content portions
 It describes for each medium
1. The specification of the elements
2. The possible access function.
3. The data coding.
 LAYOUT STRUCTURE AND LOGICAL
STRUCTURE
 Describes the co-operation of information units.
 Layout structure specifies mainly the
representation of a documents
 The logical structure includes the partitioning of
the content. 42
 Layout Structure and Logical
Structure cont…
 ODA distiinguishes the
following layout and logical
structures:
 The generic logical and
generic layout structure
includes a set of default
values
 The specific logical and
specific layout structure
describes a concrete
document.
 Figure : Relations among
content and logical and
layout structures 43
Information Architecture of ODA
 Figure : ODA information
architecture with
structure, content,
presentation and
representation model.

44
MHEG
 ISO/IEC JTC1/SC29 (Coding of Audio, Picture,
Multimedia and Hypermedia Information)
works on the standardization of the exchange
format for multimedia systems.
 The actual standards are developed at the
international level in the three working
groups cooperating with research and
industry.

45
MHEG
ISO/IEC JTC1/SC29
Coding of Audio, Picture, Multimedia
and Hypermedia Information

WG11 WG12
WG1
Coding of Moving Coding of Multimedia
Coding of Still Pictures and and Hypermedia
Pictures Associated Audio Information (MHEG)
(JBIG/JPEG) (MPEG)

Content Structure
46
MHEG
 The results of the working groups
 The Joint Photographic Expert Group (JPEG)
 The Motion Picture Expert Group (MPEG)
 These are of special importance in the area of
multimedia systems.
 The contents in the form of individual information
objects are described with the help of the above named
standards.
 The structure is specified first through timely spatial
relations between the information objects.
 The standard is the subject of the working group WG12.
 This is known as Multimedia and Hypermedia
Information Coding Expert Group (MHEG).

47
MHEG
 The final MPEG standard will be described in
three documents.
 The first part will discuss the concepts and
exchange format.
 The main concepts are covered in the first document.
 The second part part describes an alternative of
the exchange format.
 The third part should present a reference
architecture for a linkage to the script language.
 The second and third documents are still in progress.

48
MHEG
 Further discussions about MHEG are based
manly on the committee draft version, because
 All related experiences have been gained on this basis
 The basic concepts between the final standard and
this committee draft remain to be the same
 The finalization of this standard is still in progress.
 All these discussion is based on designing,
implementing and improving the MHEG
standard.

49