MULTIMEDIA COMMUNICATION

Dr. Quang Duc Tran

Interactive VOD System
Dr. Quang Duc Tran
 Introduction
• An IVOD system is capable of serving a large number of
end users to concurrently access large number of
repositories of stored data.

• IVOD is an extension of VOD. In addition to the freedom

of choosing movies, users can interact with videos and
decide the viewing schedule (tricky play, i.e., fast
forward, rewind, pause)

• Components include set-top-box at the client’s site, the

distribution network and the server
System Architecture
System Architecture
Centralized IVOD System
places processing servers
and media achieves in a
single site as a central
node.
Client
Distributed IVOD System
includes local processing
servers and media servers.
Clients’ requests are
handled by local servers.
System Architecture
Distributed IVOD System
Client
User Input Device
Visual Display
Audio Display
Network Interface
Buffers and Decoder
 The Server

Tape drive
Tape controller
Disk array (Disk Scheduling, RAID)
NAT Traversal
The overall architecture
 Characteristics
• Interactive Function (Play/Resume, Stop, Pause, Jump
Forward, Jump Backward, Fast Forward, Slow Down,
Reverse)

• Quality of Service (Fast setup time, Continuity of media

streams, Transparency of the involvement of multiple
media streams)

• Network (High speed networks, Latency, Jitter)

Introduction
Dr. Quang Duc Tran
 About ME
• Contact Information
▫ Room B1-405
▫ Department of Data Communication and Computer Network
▫ School of Information and Communication Technology
▫ Hanoi University of Science and Technology
▫ E-mail: [email protected]
▫ Tel: (+84) (4) 38682596
 Aim of this Module
• Describe the theory and operation of the major
technologies and equipment of relevance to the media
and information industries
• Introduce you to the multimedia communications and its
range of applications and networking infrastructures
• Learn about different media types (text, images, speech,
audio and video) and applications (VoIP, Video on
Demand, multimedia electronic mail, interactive
television and others)
 Syllabus
• Topic 1: Introduction to Multimedia Communication
• Topic 2: Digitalization Principles
• Topic 3: Text Processing
• Topic 4: Digital Image Processing
• Topic 5: Audio Processing
• Topic 6: Video Processing
• Topic 7: Multimedia Network
• Topic 8: Quality of Service & Synchronization
• Topic 9: RTP, RTCP & RTSP
• Topic 10: SIP & H323
 References
1. Jens-Rainer Ohm, “Multimedia Communication
Technology”, Springer-Verlag Berlin 2014.

2. William Stallings, “Data and Computer

Communication”, Prentice Hall – New Jersey 2007

3. J.D. Gibson, Editor, “Multimedia Communication”,

Academic Press, San Diego, CA, USA, 2001.

4. L.L Ball, “Multimedia Network Integration and

Management”, McGraw-Hill, 1996.
 References
5. S.J. Gibbs, and D. C. Tsichritzis, “Multimedia
Programming”, Addison-Wesley, New York, 1995.

6. W. Kou, “Digital Image Compression”, Kluwer

publishers, London 1995.

7. S.J. Solari, “Digital Video and Audio Compression”,

McGraw-Hill, 1997.
 Evaluation Method
• Final written exam (70%)
▫ Held after the end of the lecturing period
• Project/Midterm exam (30%)
▫ Written in group of 2-4 students
▫ Shall not have more than 10 pages (printed)
▫ Includes application and presentation
 Project Topics
1) Video On Demand service (Nginx)
2) Asterisk based Call Center (Asterisk)
3) Simple Video Chat Program (Socket Programming)
4) Video Conferencing System (BigBlueButton, Jitsi)
5) Media Center using KODI
6) FFMPEG: Video Codec Comparison
7) Video Streaming Server (Live555)
8) Video Recording (FFMPEG, RTSP)
9) Security: Secure RTP
INTRODUCTION
Dr. Quang Duc Tran
 Multimedia Concepts
• Multimedia:
▫ Multimedia = Integration of multimedia data (text, speech,
audio, video and images)
• Multimedia Communication is originated in a confluence
of two technological trends, i.e., Multimedia computing
and Networking.
• Applications: Video/Audio conferencing, education,
entertainment, and monitoring of remote locations, etc.
 Data vs. Signal
• To be transmitted, data must be transformed to
electromagnetic signals. Signal is continuously varying
electromagnetic wave that may be propagated over a variety
of medium, depending on spectrum.

• Data can be analog or digital. Analog data takes on

continuous values on some interval (e.g., voice and video).
Digital data takes on discrete values (e.g., text, integer).

• Signals can be analog or digital. Analog signal have an

infinite number of values in a range; Digital signals can have
only a limited number of values.
 Data vs. Signal (Cont.)
• To achieve longer distances, the analog transmission
system includes amplifiers that boost the energy in the
signal. The amplifier also boosts the noise components.
The signal becomes more and more distorted.

• For analog data, such as voice, a bit of distortion can be

tolerated. For digital data, the amplifier introduce errors.

• For analog signal, the transmission system has repeaters.

The repeater recovers the digital data from analog signal
and generate a clean analog signal.
 Periodic vs. non-periodic signal
• A periodic signal completes a pattern within a
measurable time frame, called period, and repeats such a
pattern over subsequent identical periods. One full
pattern is known as a cycle.

• A non-periodic signal changes without exhibiting a

pattern or cycle that repeats over time.

• In data communications, we commonly use periodic

analog signals and non-periodic digital signals.
 Multimedia Data
• Text: Block of characters, each represented by a fixed
number of binary digits (bit), known as codeword.
• Still images (e.g., graphics, pictures): Two dimensional
blocks of picture elements represented by a fixed number of
bits
• Sound and Video: Type of signal is known as an
analogue signal and varies continuously with time (e.g.,
telephone conversation can last for several minutes while
a movie (audio + video) can last for a number of hours).
• Animation: Sequence of graphics.
 Media Type Classification
• Captured vs. synthesized media
▫ Captured media (natural): information captured from the real
world (examples?).
▫ Synthesized media (artificial): information synthesized by the
computer (examples?).
• Discrete vs. continuous media
▫ Discrete media: spaced-based, media involves the space
dimension only (examples?).
▫ Continuous media: time-based, media involves both the space
and the time dimension (examples?)
 Media Type Classification

Sound Video Animation

Continuous Continuous

Image Text Graphics

Discrete Discrete

Captured from real world Synthesized by computer

 Text
• Plain text
▫ Unformatted
▫ Characters coded in binary form
▫ ASCII code
▫ All characters have the same style and font
• Rich text (RTF)
▫ Formatted
▫ Contains format information besides codes for characters
▫ No predominant standards
▫ Characters of various size, shape and style, e.g., bold, colorful
 Text Coding & Compression
• Text coding
▫ ASCII
 Standard code table (7 bit – 128 characters)
 Extended code table (8 bit – 256 characters)
▫ Unicode
 16 bit system (65,536 characters)
 > 110,187 graphical, formatting and control characters
• Text compression
▫ Statistical compression: Huffman code
▫ Compression using dictionary: Lempel-Ziv
▫ Compression rate: 1/2 - 2/3 document size
 Graphics
• Revisable document that retains structural information
• Consists of objects such as lines, curves, circles, etc.
• Usually generated by graphic editor of computer
programs, such as Corel Draw, Adobe Illustrator
 Still Image
• 2D matrix consisting of pixels
▫ Pixel: smallest element of resolution of the image
▫ One pixel is represented by a number of bits
▫ Pixel depth: the number of bits available to encode the pixel
• Have no structural information
• Possibly scanned and synthesized still image
Computer Synthesized
Software image

Capture and
Camera Scanned image
A/D conversion
 Still Image (Cont.)
• Examples of images
▫ Binary image: pixel depth 1
▫ Grayscale image: pixel depth 8
▫ Color image: pixel depth 24

Pixel (RGB)
 Graphics vs. Still Image
• Graphics
▫ Revisable documents
▫ Document format retains structural information
▫ Semantic content is preserved in presentation
▫ Described by objects
• Images
▫ No revisable
▫ Document format is unaware of any structural information
▫ Semantic content is NOT preserved
▫ Described as bitmaps formed of individual pixels
 Image Compression
• Lossless compression
▫ Run-Length coding (RLC)
▫ Lempel-Ziv coding
▫ GIF, BMP, TIFF
• Lossy compression
▫ Transform coding
▫ Chroma sub-sampling
▫ JPEG, JPEG2000
 Video
• Video - moving images or moving pictures
▫ Captured or Synthesized
▫ Consists of a series of bitmap images
• Frame rate: the speed to playback the video
▫ NTSC (US, Japan): 30 frames/s
▫ PAL (EU): 25 frames/s
▫ SECAM (France): 24 frames/s
▫ HDTV: 50/60 frames/s
▫ UHDTV: 120 frames/s
 Video vs. Animation
• Both images and graphics are displayed as a succession
of view, which creates an impression of movement.
• Video - moving images or moving pictures
▫ Captured or Synthesized
▫ Consists of a series of bitmap images
• Animation - moving graphics
▫ Generated by computer program
▫ Consists of a set of objects
▫ The movement of the objects can be computed
 Video Compression
• International Telecommunication Union (ITU-T)
▫ H.261: ISDN Video Phone (px64 kb/s)
▫ H. 263: PSTN Video Phone (<64 kb/s)
▫ H.26L: A variety of applications (<64 kb/s)
 Internet Video Application, VOD, Video Mail
• International Organization for Standard (ISO)
▫ MPEG-1 Video: CD-ROM (1.2 Mb/s)
▫ MPEG-2 Video: SDTV, HDTV (4-80 Mb/s)
▫ MPEG-4 Video: A variety of applications (24-1024 kb/s)
 Sound
• Sound (e.g., speech, music, noise)
▫ Analog 1-D time-based signal
▫ Vibration that propagates as a typically audible mechanical
wave of pressure and displacement through a medium such
as air or water.
• Sound wave: Sinusoidal planes wave, which has the
following properties
▫ Frequency
▫ Wavelength
▫ Amplitude
• Human ear: 20-20,000 Hz. The upper limit decreases
with age
 Digital Audio
• Audio signal that is encoded in digital form
▫ Sampling
▫ Quantization
• Sampling rate
▫ Telephone: 8 kHz
▫ CD-audio: 44.1 kHz
• Quantization
▫ Speech: 8 bit
▫ CD-audio: 16 bit
• Audio Compression
▫ Lossless compression: FLAC, Apple Lossless, MPEG-4 ALS
▫ Lossy compression: MP3 (50-60% of original size)
 A Simple Data Communication Model

Transmission
Source Transmitter Receiver Destination
System

• Source generates the data to be transmitted (Examples?)

• Transmitter transforms and encodes the information to
produce electromagnetic signals that can be transmitted
across some sort of transmission system.
• Transmission system can be transmission line or a complex
network connecting source and destination
• Receiver accepts the signals and convert them into a form that
can be handled by the destination device.
• Destination takes the incoming data from the receiver.
 Communication Tasks
Transmission system utilization Flow control
Interfacing Addressing
Signal generation Routing
Synchronization Recovery
Exchange management Security
Error detection and correction Network management
 Distributed Multimedia System
• Distributed Multimedia System involves transmission
and distribution of multimedia information on the
network.

• Multimedia Servers
▫ Silicon Graphic
▫ HP Media Server
• Multimedia Networks
▫ PSTN/Data Network etc.
• Multimedia Clients
Distributed Multimedia System (Cont.)
• Live media transmission system
▫ Capture, compress, and transmit the media on the fly
(example?)
• Send stored media across the network
▫ Media is pre-compressed and stored at the server. This
system delivers the stored media to one or multiple
receivers (example?)
• Differences between the two systems
▫ Media capture: Real-time media capture vs. pre-stored
media.
▫ Media compression: Real-time/off-line compression?,
compression can be adjusted during transmission?
Distributed Multimedia System (Cont.)
• Requirements of networked multimedia applications
▫ Delay requirements
▫ Quality requirements
 Satisfactory quality of media presentation
 Synchronization requirement
 Continuous requirement
 Can tolerate some degree of information loss
• Challenges of distributed multimedia system
▫ Conflict between media size and bandwidth limit
▫ Conflict between the user requirement of multimedia
application and the best-effort network.
▫ How to meet different requirement of different users?
 Multimedia Server
Tape storage
• Multimedia Servers
▫ Data storage and retrieval CPU
Tape
▫ Media synchronization Controller
▫ Video/Audio Compression System bus
• Requirements
▫ High performance Network
Disk Network
▫ High reliability Controller adapter
High-speed
bus
MPEG-1: 1-2 Mb/s
Disk on-line MPEG-2: 2-40 Mb/s
storage
 Multimedia Equipments
• Digital equipments:
▫ Computers
▫ A/D, D/A converters for Video and Audio.
• Analogue equipments:
▫ Video camera,
▫ Microphone, Speaker,
▫ Tape player/recorder.
• Synchronization equipments:
▫ Support time synchronization.
• Interactive equipments:
▫ Display, mouse, keyboard etc.
 Multimedia Networks
• Telephone Networks:
▫ Public switched telephone networks (PSTNs): initially
designed to provide speech services, now can support
multimedia applications due to the advances in Digital
Signal Processing hardware and software.
• Data Networks: initially supported data applications (e-
mail/ftp), now support much complex multimedia
applications.
• Broadcast Television Networks: Broadcast TV
• Broadband Multiservice Networks: Multiservice.
 Multimedia Operating System
• Multimedia operating system provides a comfortable
environment for the execution of programs, and ensures
effective utilization of the computer hardware.
Multimedia Applications

Multimedia I/O Stream Multimedia Device

Manager Handlers Manager

Physical Device Stream and Multimedia I/O Drive

Driver Synchronization manager Audio/ video codec

Hard disc Optical CD ROM Analog video Analog

drive drive drive NTSC, PAL audio
PCM Monitor
Multimedia Operating System (Cont.)
• MMOS Requirements:
▫ Soft real-time applications: statistical guarantees
▫ Interactive applications: no absolute performance
guarantees, but low average response times
▫ Throughput-intensive application: no performance
guarantees, but high throughput.
▫ Fair, Proportionate resource allocation: Divide
resources according to application requirements.
▫ Application Isolation: Preventing misbehaving or
overloaded application from effecting others (e.g.,
overloaded web server should not affect streaming media
server).
 Multimedia Operating System (Cont.)
Root Root

I TP SRT

Class-specific Schedulers
I SRT TP Http SRT

33% 33% 33% 33% 66%

Audio Video D1 D2 Audio Video C Class-Independent

Scheduler

20% 80% 50% 50% 20% 80%

FCFS

CPU Scheduler Packet Scheduler Disk Scheduler

 Multimedia Applications
• Multimedia Information Systems: Multimedia
Database, Information hypertexts, Hypermedia,
Electronic books, Multimedia expert systems, etc.

• Multimedia Communication Systems: VOIP,

Audio-video Communication, Computer-supported
collaborative works, Videoconferencing, IPTV,
Streaming media and Multimedia Teleservices, etc.

• Multimedia Entertainment Systems: Game, 3D

computer games, Multimedia design, Multiplayer
networks, Interactive audiovisual productions, etc.
 Multimedia Applications (Cont.)
• Multimedia Educational Systems: E-Books, E-
learning, Flexible teaching materials, Simulation
education systems, etc.

• Multimedia Business Systems: Electronic

commerce, Marketing, Multimedia presentation –PR,
Virtual shopping, etc.
Multimedia Application Environment
 Example (Video on Demand)
ADSL

STB

Media Server Authentication

/Billing Server

CATV

STB
Router
Streaming Server

Modem

STB
Web Server