Multimedia Data Representation

Issues to be covered:
• Digital Audio
– Sampling Theorem
– Digital Audio Signal Processing/Audio Effects
– Digital Audio Synthesis
– MIDI — Synthesis and Compression Control
• Graphics/Image Formats
– Colour Representation/Human Colour Perception
• Digital Video
– Chroma Subsampling

B. Okuku 1
General Themes across all above

• Sampling/Digitisation
– Sampling Artifacts — Aliasing
• Compression requirements
– Data formats especially size
– Human Perception!compression ideas
•Compression Algorithms

B. Okuku 2
Digital Audio

What is Sound?
Source — Generates Sound
• Air Pressure changes
• Electrical — Loud Speaker
• Acoustic — Direct Pressure Variations

Destination — Receives Sound

• Electrical — Microphone produces electric signal
• Ears — Responds to pressure hear sound (MPEG
Audio)

B. Okuku 3
Digitising Sound
• Microphone produces analog signal
• Computer like discrete entities
Need to convert Analog-to-Digital — Specialised
Hardware
Also known as Sampling

B. Okuku 4
Digital Sampling

Sampling basically involves:

• Measuring the analog signal at regular discrete
intervals
• Recording the value at these points

B. Okuku 5
Sample Rates and Bit Size

•How do we store each sample value (Quantisation)?

8 Bit Value (0-255)
16 Bit Value (Integer) (0-65535)

•How many Samples to take?

11.025 KHz — Speech (Telephone 8 KHz)
22.05 KHz — Low Grade Audio
(WWW Audio, AM Radio)
44.1 KHz — CD Quality

B. Okuku 6
Nyquist’s Sampling Theorem
Sampling Frequency is very important in order to
accurately reproduce a digital version of an Analog
Waveform.

Nyquist’s Theorem:
The Sampling frequency for a signal must be at least
twice the highest frequency component in the signal.

B. Okuku 7
Figure 1: Sampling at Signal Frequency

B. Okuku 8
Figure 2: Sampling at Twice Nyquist Frequency

B. Okuku 9
Figure 3: Sampling at above Nyquist Frequency

B. Okuku 10
Implications of Sample Rate and Bit Size

Affects Quality of Audio

• Ears do not respond to sound in a linear fashion
(MPEG Audio)
• Decibel (dB) a logarithmic measurement of sound
• 16-Bit has a signal-to-noise ratio of 98 dB — virtually
inaudible
• 8-bit has a signal-to-noise ratio of 50 dB
• Therefore, 8-bit is roughly 8 times as noisy
– 6 dB increment is twice as loud
• Click Here to Hear Sound Examples

B. Okuku 11
Implications of Sample Rate and Bit Size (cont)

Affects Size of Data

B. Okuku 12
Practical Implications of Nyquist Sampling Theory

Must low pass filter signal before sampling:

Analog low pass filter used as signal is not yet digitised.

Otherwise strange artifacts from high frequency signals
would appear — Aliasing

B. Okuku 13
Why are CD Sample Rates 44.1 KHz?

Upper range of human hearing is around

20-22 KHz — Apply Nyquist Theorem

Emerging Theme throughout this course:

• Perceptual traits of Human Auditory system.
• Don’t both recording data above 22 KHz
• So filter them out — low pass filter.

B. Okuku 14
Common Audio Formats
• Popular audio file formats include
– .au (Origin: Unix, Sun),
– .aiff (MAC, SGI),
– .wav (PC, DEC)
• Compression can be utilised in some of the above but
is not Mandatory
• A simple and widely used (by above) audio
compression method is Adaptive Delta Pulse Code
Modulation (ADPCM).
– Based on past samples, it predicts the next sample
and encodes the difference between the actual value
and the predicted value.

B. Okuku 15
Common Audio Formats (Cont.)
• Many formats linked to audio applications
• Most use some compression
• Common ones:
– Sounblaster — .voc (Can use Silence Deletion (More on this
later (Audio Compression))
– Protools/Sound Designer – .sd2
– Realaudio — .ra.
– Ogg Vorbis — .ogg
– AAC , Apple, mp4 — More Later
– Flac — .flac, More Later
– Dolby AC coding — More Later
• MPEG AUDIO— More Later (MPEG-3 and MPEG-4)

B. Okuku 16
Synthetic Sounds —reducing bandwidth?

• Synthesise sounds — hardware or software

• Client produces sound — only send parameters to control sound
(MIDI/MP4 later)
• Many synthesis techniques could be used, For example:
– FM (Frequency Modulation) Synthesis – used in low-end Sound
Blaster cards, OPL-4 chip, Yamaha DX Synthesiser range popular in
Early 1980’s.
– Wavetable synthesis – wavetable generated from sampled
sound waves of real instruments
– Additive synthesis — make up signal from smaller simpler
waveforms

B. Okuku 17
– Subtractive synthesis—modify a (complex) waveform
but taking out (Filtering) elements
– Granular Synthesis — use small fragments of existing
samples to make new sounds
– Physical Modelling — model how acoustic sound in
generated in software
– Sample-based synthesis—record and play back
recorded audio, often small fragments and audion
processed.
• Most modern Synthesisers use a mixture of samples
and synthesis.

B. Okuku 18