MMC Unit II

By
M.C.Aralimarad
Text and Image Compression in Multimedia Applications

• Introduction to Compression (Section 3.1)

• Purpose of Compression:
• Reduce the volume of information for transmission or storage.
• Especially important for text, images, speech, and audio.
• Bandwidth Considerations:
• Compression reduces required bandwidth, leading to cost savings, especially on networks with
call duration-based charges.
• Applications:
• Widely used across multimedia types to facilitate efficient data transfer and storage.
• Compression Principles (Section 3.2)
• Four key principles to understand compression algorithms:
• Source Encoders and Destination Decoders
• Lossless and Lossy Compression
• Entropy Encoding
• Source Encoding
Source Encoders and Destination Decoders (Section 3.2.1)
• Purpose:
• Compress the source information before transmission.
• Apply a matching decompression algorithm at the destination to reconstruct the original or approximate data.
• Encoding and Decoding Process:
• Source Encoder: Compresses the data before sending it.
• Destination Decoder: Decompresses the data upon receipt.
• Implementation in Systems:
• In some applications, compression and decompression must happen in separate devices for performance reasons.
• Example: Multimedia applications where one system encodes (compresses) data while another system decodes
(decompresses) it.
• Lossless and Lossy Compression (Section 3.2.2)
• Lossless Compression:
• Retains the original information entirely, allowing exact reconstruction of the source data.
• Commonly used in scenarios where data integrity is critical, such as text and some types of image files.
• Examples: File transfer over a network or data archiving.
• Lossy Compression:
• Some information is permanently lost during compression, resulting in an approximation of the original data.
• Often used for images, audio, and video, where exact replication is less crucial, and lower data size is prioritized.
• Goal: Minimize storage or transmission requirements while maintaining acceptable quality.
Source Encoder/Destination Decoder
Compression Techniques: Text and Image Compression in Multimedia

Source Encoder/Destination Decoder Configurations

• Purpose: Compress and decompress data for efficient transmission over a network.
• Figure 3.1: Encoder/Decoder Configurations:
• (a) Software Only: Both encoding and decoding performed using software applications on source and
destination computers.
• (b) Special Processors/Hardware: Dedicated hardware used for compression/decompression to enhance
performance.
• Applications: Used in systems where either exact data reproduction or approximation is
acceptable, depending on user requirements.
Lossy Compression in Multimedia
• Characteristics: Produces a version that is an approximation of the original data, perceived as a
true copy by the user.
• Use Cases: Often applied to digitized images, audio, and video where exact accuracy is less
critical.
• Human eye and ear sensitivity allows for minor data losses that are undetectable.
• Advantages:
Entropy Encoding (Section 3.2.3)
•Definition:
•A type of lossless compression focused on efficiently representing data by using coding schemes.
•Types of Entropy Encoding:
•Run-Length Encoding:
•Useful for compressing strings with long substrings of the same character or binary digits.
•Encodes a string by specifying the character and the count of its occurrences.
•Example: A sequence of 0000001111111000 can be encoded as (0,7), (1,10), (0,5), (1,2) to
reduce data size.
•Statistical Encoding:
•Uses codewords of varying lengths based on character frequency.
•Characters that appear more frequently receive shorter codewords, while less frequent
characters get longer codewords.
•Often applied to ASCII characters, assigning shorter codes to commonly used characters to save
space.
•Advantages:
•Effective for reducing redundancy in data without loss, making it suitable for transmitting ASCII-based
information.
Compression Techniques: Entropy, Source, and Transform Encoding

• Entropy in Data Compression

• Definition:
• The minimum average number of bits required to represent a source symbol.
• Shannon’s Entropy Formula:

• Where:
• nnn = number of different symbols.
• PiP_iPi​= probability of occurrence of symbol iii.

• Average Bits per Codeword:

• Calculated using

representing the average efficiency of an encoding scheme.

• Source Encoding (Section 3.2.4)
• Purpose:
• Produces an alternative representation of source information to optimize for
compression.
• Types of Source Encoding:
• Differential Encoding:
• Used when data has a large range of values with small differences between successive values.
• Encodes only the difference between successive values, which saves bandwidth.
• Example:
• If maximum amplitude variation requires only 3 bits instead of 12, this approach achieves a 75% bandwidth
reduction.
• Can be either lossy or lossless, depending on the number of bits used to encode the difference values.
• Transform Encoding:
• Converts data from one format to another for more effective encoding.
• Often used in conjunction with other compression methods to enhance data representation and reduce
redundancy.
1.Need for Compression
Compression and Spatial Frequency

1. Large multimedia files require compression to reduce data size.

2. Achieved by transforming data into a form that minimizes redundancies.
2.Spatial Frequency in Images
1. Refers to the rate of pixel value changes across an image.
2. Low spatial frequency: Slow change in pixel values (smooth areas).
3. High spatial frequency: Rapid changes in pixel values (edges and details).
3.Human Eye Sensitivity
1. Human vision is more sensitive to lower spatial frequencies.
2. High spatial frequencies can be selectively reduced in compression.
• Example of Pixel Patterns (Figure 3.2(a))
1.Pixel Patterns
1. Two sample patterns show horizontal spatial frequency components.
2. Pattern 1: Alternating black and white pixels, higher frequency components.
3. Pattern 2: Larger blocks of consistent colors, lower frequency components.
2.Amplitude and Spatial Frequency
1. Patterns illustrate how amplitude varies with spatial frequency.
 D is c re te C o s in e T ra n s fo rm (D C T )P rin c ip le s

1.DCT Transformation (Figure 3.2(b))

1. DCT (Discrete Cosine Transform) converts spatial pixel values into frequency components.
2. Each pixel block is transformed into a matrix representing spatial frequencies.
2.Compression Using DCT
1. Low-frequency components hold most image details, while high-frequency components can be reduced or removed.
2. Reducing high-frequency components allows efficient compression without significant quality loss.
3.Matrix Transformation
1. The pixel matrix is transformed into a 2D frequency matrix.
2. Rows and columns of the matrix show horizontal and vertical frequency components.
• Benefits of Using DCT for Image Compression
1.Lossless vs. Lossy Compression
1. DCT is nearly lossless; only minor data rounding errors may occur.
2. Applying a threshold on amplitude can selectively drop higher frequencies, making it lossy but effective.
2.Data Reduction and Storage Efficiency
1. Using DCT allows for reduced data without degrading perceived image quality.
2. Frequently used in formats like JPEG for image compression.
3.Image Quality Maintenance
1. By retaining low-frequency components, the perceived quality of the image remains high even after compression.
1.Types of Text
Text Compression Overview

1. Unformatted, formatted, and hypertext – each represented as strings of characters.

2. Text compression must retain the information exactly, so lossless compression is essential.
2.Text Compression Techniques
1. Based on statistical encoding to reduce redundancy.
2. Two primary types:
1. Single-character codewords (e.g., Huffman coding).
2. Variable-length codewords (e.g., Lempel-Ziv (LZ) algorithms).
• Statistical Encoding Methods
1.Purpose of Statistical Encoding
1. Designed for efficient text compression based on character frequency.
2. More common characters use shorter codewords.
2.Static vs. Adaptive Coding
1. Static Coding: Codewords are determined in advance and used throughout.
2. Adaptive Coding: Codewords are updated dynamically based on character frequency during transmission.
3.Common Methods
1. Huffman Coding: Assigns variable-length codes based on frequency.
2. Arithmetic Coding: An example of adaptive coding, where intervals represent different characters.
Static Huffman Coding Principles (Section 3.3.1)
1.Character Analysis
•Analyzes characters to determine frequency.
•Constructs a binary tree with shorter codes for more
frequent characters.
2.Huffman Tree Construction (Figure 3.3(a))
•A binary tree with each branch as 0 or 1.
•Root Node (RN): Starting point.
•Branch Nodes (BN): Intermediate nodes.
•Leaf Nodes (LN): Final nodes with the assigned
codewords.
(Figure 3.3(a))
Example of Huffman Tree (Figure 3.3(b))
1.Tree Derivation Process
•Nodes are ordered by frequency of occurrence.
•Nodes with higher frequencies are closer to the root, resulting
in shorter codewords.
2.Tree Example
•Character Order: Based on frequency (e.g., D, C, B, A).
•Final codewords generated by tracing from the root to each
leaf node.
3.Resulting Codes
•Each character has a unique binary code:
•A = 1
•B = 01
•C = 001
•D = 000
Huffman Code Construction for Compression
1.Goal of Huffman Coding
•To assign shorter binary codewords to frequently
occurring characters to minimize
•the total number of bits required for storage or
transmission.
2.Example Code Calculation
•For a string such as "AAAABBCD," the total bit length is
calculated using the assigned codewords:
•4 x 1 + 2 x 2 + 1 x 3 + 1 x 3 = 14 bits
Steps in Huffman Code Tree Construction (Ref: Figure 3.3)
1.Determining Character Frequency
•Characters are listed in decreasing order of frequency.
•Example: In Figure 3.3(b), characters are ordered as
D1, C1, B2, and A4.
2.Building the Tree
•Initial Leaf Nodes: Two least frequent characters
are assigned to branches.
•Branch Node Creation: Sum of weights creates
a new branch node.
•Repeat this process until only two nodes remain,
forming the tree structure.
 Introduction to Huffman Encoding

• Huffman Encoding:
• A lossless data compression algorithm.
• Assigns shorter codes to more frequent characters, and longer codes to less frequent
characters.
• Efficient for compressing text and image data.
Huffman Encoding Example
•Figure 3.4 (Huffman Encoding Example):
•Code Generation Process:
•Example characters with probabilities:
•A = 0.25, B = 0.25, C = 0.14, D = 0.14, E = 0.11, F = 0.055, G = 0.055, H = 0.055
•The Huffman tree is constructed by combining the lowest-probability nodes.
•Codewords Derived:
•Shorter codes for frequent characters, e.g., A = 1, B = 01
•Longer codes for less frequent characters, e.g., F = 00001, G = 00000
• Huffman Code Tree
• Figure 3.4(b) (Huffman Code Tree):
• Visual representation of how Huffman codes are derived.
• Nodes combined in ascending order of weight:
• Leaf nodes: Represent characters with weights (probabilities).
• Internal nodes: Sum of the probabilities of connected nodes.
• The tree structure ensures the prefix property:
• No code is a prefix of another, ensuring unique decodability.
Huffman Decoding Process
•Figure 3.5 (Decoding Process):
•Decoding Flowchart (Figure 3.5(a)):
•The bitstream is processed sequentially to identify valid codewords.
•Steps:
1.Retrieve bits from the BIT-STREAM.
2.Append bits to CODEWORD variable.
3.Check if CODEWORD matches a valid code.
4.If yes, write ASCII character to RECEIVE_BUFFER; if no, continue.
•Example Decoding (Figure 3.5(b)):
•Decoding a bitstream: 10111000001…
•Extracted characters: A, B, B, A, D
•Shows sequential decoding based on valid codewords.
• Key Properties of Huffman Codes
• Prefix Property:
• Ensures that no codeword is a prefix of another.
• Prevents ambiguity during decoding; each codeword can be uniquely
decoded.
• Efficiency:
• Huffman codes minimize the average code length, optimizing storage and
transmission efficiency.
• Applications:
• Widely used in text compression (e.g., ZIP files) and image formats (e.g.,
JPEG).
• Dynamic Huffman Coding:
Introduction to Dynamic Huffman Coding

• A variation of Huffman coding where codewords are updated as new characters are transmitted.
• Eliminates the need for a predefined frequency table at the start.
• Allows efficient, adaptive compression, especially useful when data characteristics change over
time.
• Mechanism of Dynamic Huffman Coding
• Operation:
• Both transmitter and receiver start with an initial tree structure, consisting of a root node and a
single empty leaf node (with zero frequency).
• As each character is encountered, its frequency is updated, and the tree is adjusted.
• Codewords adapt in real time as characters are transmitted, with the tree structure modified
dynamically at both the transmitter and receiver.
• Advantages of Dynamic Huffman Coding
• Advantages:
• Reduces overhead by adapting the code structure to the incoming data stream.
• No need to transmit a frequency table beforehand; the receiver can reconstruct the tree on-the-fly.
• Useful for data with varying frequency distributions, enabling efficient, real-time compression.
 Dynamic Huffman Encoding Algorithm

Tree Construction Steps:

•Initial State (a): The tree starts with a root node and a single empty leaf (for unknown characters).
•Updating the Tree:
•(b) First character (a) added, updates frequency, and creates new nodes.
•(c) Second character (b) processed; tree is restructured to reflect updated frequencies.
•(d) Subsequent characters continue to update frequencies, maintaining tree structure.
Key Observations:
•The tree grows as characters are processed, with higher-frequency characters having shorter codewords.
•The receiver mirrors this process, allowing accurate decoding without predefined tables.
• Applications and Considerations
• Applications:
• Dynamic Huffman coding is ideal for text and data streams where character frequencies change dynamically.
• Commonly used in applications like live data compression and streaming, where efficiency and adaptability are crucial.
• Considerations:
• While efficient, dynamic Huffman coding requires both encoder and decoder to manage and synchronize the changing
tree structure in real time.
• Works best in environments with controlled latency to ensure seamless tree updates.
• Introduction to Dynamic Huffman Coding
• Definition: A variation of Huffman coding where the coding tree adapts dynamically.
• Purpose: Eliminates the need to know the data’s frequency distribution in advance.
• Mechanism: Transmitter and receiver construct and update the Huffman tree as data is
transmitted and received.
• Process of Dynamic Huffman Coding
• Initial Setup:
• Both transmitter and receiver start with a tree containing only a root node and an empty leaf
node.
• This empty leaf node is assigned a zero frequency of occurrence.
• Tree Construction:
• As each character is transmitted, the tree is updated based on its occurrence
• Encoding Steps (Figure 3.6 Example)
• Step 1: Transmit first character in uncompressed form if it’s the first occurrence.
• Step 2: Update tree structure by:
• Adding the character to the tree.
• Reassigning empty leaf nodes for future additions.
• Step 3: For subsequent characters:
• Check if character is already in the tree.
• Tree Update Rules
• Frequency Adjustment:
• Increment the frequency of each node when a character is added.
• Maintain tree balance by ensuring nodes are ordered by weight.
• Node Reordering:
• Swap nodes if necessary to maintain tree structure.
• Consistency: Both encoder and decoder update their trees identically
• Example of Encoding Sequence
• Illustrated Steps (Figure 3.6):
• Demonstrates character addition and code generation.
• Shows tree reconfiguration as new characters are added or repeated.
• Highlights incremental tree modifications
• Advantages and Applications
• Advantages:
• Reduces need for prior data frequency analysis.
• Efficient for real-time data encoding where frequencies change over time.
• Applications: Used in text compression, especially for adaptive systems where
Arithmetic Coding Overview
• Definition: Arithmetic coding achieves the Shannon value optimally, unlike
Huffman coding which is effective only if symbol probabilities are integer powers
of 1/2.
• Advantage: Produces a single codeword for each encoded string instead of
separate codewords for each character, leading to better compression.
• Complexity: More complex than Huffman coding, so the discussion is limited to
basic static coding.
• Example of Arithmetic Coding
• Message Structure: The example encodes the string "went." with probabilities
assigned to each character:
• e = 0.3
• n = 0.3
• t = 0.2
• w = 0.1
• . (termination character) = 0.1
• Encoding Process:
• A range (0 to 1) is divided based on character probabilities.
• Range Segmentation for Characters
• Character Segmentation:
• Each character has a segment on the range 0 to 1:
• e: 0.0 to 0.3
• n: 0.3 to 0.6
• t: 0.6 to 0.8
• w: 0.8 to 0.9
• .: 0.9 to 1.0
• Process: Encoding starts by narrowing down segments within the assigned ranges for
each character
• Encoding "went." Example in Steps
• Character "w" Encoding:
• "w" starts in the range 0.8 to 0.9.
• Subdivide 0.8 to 0.9 based on character probabilities.
• New range for "e" becomes 0.8 to 0.83, and so on.
• Continued Segmentation:
• Each character narrows down the range further, creating progressively smaller segments.
• Final Range: The codeword for "went." lies in the range 0.81602 ≤ codeword < 0.8162
• Decoding Process for Arithmetic Coding
• Decoding Steps:
• Use the codeword range to backtrack and decode the character string.
• Identify each character segment based on cumulative probabilities.
• Example: A codeword in range 0.8161 corresponds to the first character as "w".
• Iterative Process: Continues until reaching the termination character
• Characteristics of Arithmetic Coding
• Precision Requirements:
• Number of decimal digits in codeword increases with message length.
• Precision affects the maximum number of characters in a single encoded message.
• Applications: Useful in scenarios where floating-point precision is manageable
in both encoding and decoding systems.
 Introduction to Lempel-Ziv Coding (LZ Coding)

• LZ Compression Basics:
• A dictionary-based compression technique.
• Uses strings of characters rather than single characters.
• Encoding Method:
• Encoder sends the index of a word's location in the dictionary.
• Decoder reconstructs text based on these indices.
• LZ Coding and Practical Applications
• Applications in Word Processing:
• Many word processors use LZ for compression.
• Dictionary contains approximately 25,000 words.
• Compression Efficiency:
• Example: The word "multimedia" requires only 15 bits with a dictionary, compared to 70
bits with ASCII encoding.
• Compression Ratio: Achieves a ratio of 4.7:1, where shorter words achieve lower ratios.
• Lempel-Ziv Algorithm (Example 3.3)
• Dictionary-Based Compression:
• Uses a dictionary with 4096 entries for text compression.
• Compression ratio example: 3.5:1 for ASCII codewords.
• Efficiency:
• Higher efficiency with a dynamic dictionary approach.
• Lempel-Ziv–Welch (LZW) Coding
• Dynamic Dictionary Construction:
• Dictionary builds up during encoding.
• Both encoder and decoder update dictionary simultaneously.
• Key Idea:
• Reduces redundancy by storing repeated sequences.
• Example sequence: "This is simple as it is…".
• Slide 9: LZW Coding Process (Fig 3.8)
• Basic Operation:
• Initial dictionary contains character set (e.g., ASCII).
• New words added dynamically (e.g., "This", "is", "simple").
• Dynamic Expansion:
• Incremental indexing for added words.
• Extended LZW Dictionary (Fig 3.8b)
• Dictionary Expansion:
• Dictionary expanded to accommodate new sequences.
• Efficient storage and retrieval using dynamic indexing.
• Example:
• Dictionary initially starts with individual characters.
• Extends with multi-character sequences as compression progresses.