IIIUS00554.

4286A
IIII
United States Patent (19) 11) Patent Number: 5,544,286
Laney (45) Date of Patent: Aug. 6, 1996
54) DIGITAL VIDEO DATA COMPRESSION 5,047,853 9/1991 Hoffert et al. .......................... 358/133
TECHNIQUE 5,353,061 10/1994 Rodriguez et al. ..................... 348/409
Primary Examiner-Raymond J. Bayer
I75) Inventor: Stuart T. Laney, Seattle, Wash. Assistant Examiner-Gabriel I. Garcia
73) Assignee: Microsoft Corporation, Redmond, Attorney, Agent, or Firm-Seed and Berry
Wash. (57) ABSTRACT
(21) Appl. No.: 11,317 A technique for compressing digital video data provides
improved compression over conventional block compres
22 Filed: Jan. 29, 1993 sion techniques. In this technique, image data is broken
6 down into cells and iteratively compressed. The cells are
3. s - - - - - - - - - - - - - -a - Josis, compressed using compression formats that are most appro
(52) irr priate for the contents of the cells. A determination is first
58) Field of Search .................................... 395/109, 114; made whether a cell is substantially identical to a succeeding
358/261.1, 261.2, 261.3, 426, 466 cell in a previous frame. If the cell is substantially identical
f Cited to the cell in the previous frame, the cell is encoded in
56) References Cite compressed form as a duplicate of the previous cell. More
U.S. PATENT DOCUMENTS over, solid-color compression approaches, two-color com
pression approaches and eight-color compression
22.
s F 1 -/ 2.E. Gr as a reel KSe o 3. approaches may be integrated into the compression tech
4,953,023 8/1990 Kondo ............ "35 mique.
5,041,916 8/1991 Yoshida et al. . ... 358/433
5,046,119 9/1991 Hoffert et al. ............................ 382/56 20 Claims, 26 Drawing Sheets

Get Cell Frona 116

Previous Frame

Expand Ce 118
if Necessary

Calculate Color
Difference Between 120
Current Cell and Cell
From Previous Frane

Is Color
Difference> Skip
Cell Threshold

Designate as a Designate as Not 126

Skip Cell a Skip Cell
U.S. Patent Aug. 6, 1996 Sheet 1 of 26 5,544,286

Get Next
Cellin Image

14

(Prior Art)
Figure 1
U.S. Patent Aug. 6, 1996 Sheet 2 of 26 5,544,286

Calculate
Luminances for 16
Pixels in the Cell

Determine Min and

Max of Luminances
of Pixels in the Cell

Threshold

(Prior Art)
Figure 2
U.S. Patent Aug. 6, 1996 Sheet 3 of 26 5,544,286

Calculate Average
Luminance for 22
Pixels in the Cell

Any
Pixels in the
Cell Left

Get Luminance 26
for Next Pixel ---

3() 32
IsS. N 28
Mark Pixel as Next A"

Being in the Set NO ixel Luminance N. YES Mark Pixel as

of Dimmer Pixels
--- Average Being l the Set
Luminance of Brighter Pixels

(Prior Art)
Figure 3
U.S. Patent Aug. 6, 1996 Sheet 4 of 26 5,544,286

Calculate Average
Color for the Set
of Brighter Pixels

Assign the Average

Color to Each
Pixel in the Set
of Brighter Pixels

Calculate Average
Color for the Set
of Dimmer Pixels

Assign the Average

Color to Fach
Pixel in the Set
of Dimmer Pixels

(Prior Art)
Figure 4
U.S. Patent Aug. 6, 1996 Sheet 5 of 26 5,544,286

Divide 4x4 Cell 42

into 42x2 Cells

Any
2x2 Cells
Left

YES

Get Next 46
2x2 Cell

(Prior Art)
Figure 5
U.S. Patent Aug. 6, 1996 Sheet 6 of 26 5,544,286

15 O

Bit Mask - 48

15 14 109 54 O

Greer 50

15 14 109 54 O

Green 52

(Prior Art)
Figure 6a
U.S. Patent Aug. 6, 1996 Sheet 7 of 26 5,544,286

4(01+4\), aq9un61-I
99^

øn?EI

I
0
6 ||
0
6
I.3 $)
|
S
U.S. Patent Aug. 6, 1996 Sheet 8 of 26 5,544,286
U.S. Patent Aug. 6, 1996 Sheet 9 of 26 5,544,286
U.S. Patent Aug. 6, 1996 Sheet 10 of 26 5,544,286

i
U.S. Patent Aug. 6, 1996 Sheet 11 of 26 5,544,286

Calculate Solid
Color Threshold
From Base Threshold

Calculate Skip Cell

Threshold From
Basic Threshold

Figure 10a
U.S. Patent Aug. 6, 1996 Sheet 12 of 26 5,544,286

Change Quality
level to a
New Level

Recalculate
Thresholds. Using the
New Quality Level

Figure 10b
U.S. Patent Aug. 6, 1996 Sheet 13 of 26 5,544,286

Compress Into Compress into 96

8-Bit Format 16-Bit Format

Figure 11
U.S. Patent Aug. 6, 1996 Sheet 14 of 26 5,544,286

Encode as a
Skip Cell

Encode as a
Solid-Color Cell

Encode as a
Two-Color Cell

Encode as an
Eight-Color Cell

Figure 12
U.S. Patent Aug. 6, 1996 Sheet 15 of 26 5,544,286

Get Cell From

-- 1--
Previous Franc 1166

Expand Cell 18
if Nccessary

Calculate Color
Difference Between 1-1
20
a
Current Cell and Cell
From Previous Frame

22
Is Color
Difference > Skip
Cell Threshold

Designate as a Designate as Not --126

Skip Cell a Skip Cell

Figure 13
U.S. Patent Aug. 6, 1996 Sheet 16 of 26 5,544,286

Þjaun61-I
#|1 0|goonsdI!?X|º0So |
ISI6()
U.S. Patent Aug. 6, 1996 Sheet 17 of 26 5,544,286

Calculate Average 30
Color of Pixels
in Cell

Build Cell Having

All Pixels With 132
the Avcrage Color

Calculatc. Total
Color Difference 134
Between Cell and
Built Cell

Solid-color
Threshold
t

Designate as a Designate as Not 140

Solid-Color Cell a Solid-Color Ceil

Figure 15
U.S. Patent Aug. 6, 1996 Sheet 18 of 26 5,544,286

~
I
##

0# sw
|

e9,aun61-I {9}6[ q
a un961?-I
U.S. Patent Aug. 6, 1996 Sheet 19 of 26 5,544,286

Calculate Average
Color of Pixels and
Build a Cell Having 46
Only Pixels With
the Average Color

Calculate the Color

Difference Between 148
the Cell and
the Built Cell

150
Is
Error - Base
Threshold
i)

Designate as a
Two-Color Cell

Figure 17
U.S. Patent Aug. 6, 1996 Sheet 20 of 26 5,544,286

re
r
H

f
& S.
s C
"S
C O
U.S. Patent Aug. 6, 1996 Sheet 21 of 26 5,544,286

(Z9||
IGI007
JO;
„
„? „Ó,JO?J? ?C) †79I
09]

3q-iºnfpl-.f

?I
U.S. Patent Aug. 6, 1996 Sheet 22 of 26 5,544,286
U.S. Patent Aug. 6, 1996 Sheet 24 of 26 5,544,286

15

5
a
Color for "1" in Cell () - 186

1. 5

5
currica
curren
circa
Color for "O" in Cell () 188

() Color for "l" in Cell 1 190

5 O
O Color for "O" in Cell --- 192

5
Color for "1" in Cell 2 194

Color for "O" in Cell 2 96

15

Color for "l" in Cell 3 198

o
cavaca
Color for "O" in Cell 3 200

Figure 20b
U.S. Patent Aug. 6, 1996 Sheet 25 of 26 5,544,286

DocS 204
First Bit = "O"
Dccompress as a
Two-Color Cell
?

Does Decompress as an
Third Bit = '1" Eight-Color Cell
)

Do
First Six Bits = Decompress as a
"100001" Skip Cell

Decompress as a
Solid-Color Cell

Figure 21a
U.S. Patent Aug. 6, 1996 Sheet 26 of 26 5,544,286

Do First 220
Six Bisc "10000"
Decompress as
a Skip Cell

Decompress as a
Solid-Color Ceil

Sign, Bit of First Decompress as a

Color Word a "1" Two-Color Cell
s

Decompress as an
Eight-Color Cell

Figure 21b
 5,544,286
1. 2
DIGITAL VIDEO DATA COMPRESSION is marked as being in the set of brighter pixels (step 32).
TECHNIQUE Steps 24, 26, 28, and 30 or 32 are then repeated for the
remaining pixels in the cell. When all of the pixels have been
TECHNICAL FIELD examined, the steps shown in the flowchart of FIG. 4 are
performed.
The present invention relates generally to data processing In accordance with the steps shown in FIG. 4, the average
systems and, more particularly, to a digital video data color for the set of brighter pixels is calculated (step 34). The
compression technique. average color is then assigned as the color of each of the
BACKGROUND OF THE INVENTION
pixels in the set of brighter pixels (step 36). The assignment
10 of the average color to the pixels in the set of brighter pixels
Many different classes of digital video data compression is realized by manipulating a bit mask for the cell (see 48 in
techniques are used to compress digital video data. Block FIG. 6a and 54 in FIG. 6b), as will be described below. The
compression techniques are a class of techniques which average color for the set of dimmer pixels is, likewise,
compress digital video data on a block-by-block basis. A calculated (step 38). The average color for the set of dimmer
block (also known as a 'cell') refers to a fixed-size group of 15
pixels is then assigned as the color of each of the pixels in
pixels. A cell is typically formed by a two-dimensional the set of dimmer pixels (step 40). As a result, each four
pixels-by-four-pixels cell has two colors assigned to its
symmetric group of pixels, such as two pixels by two pixels, pixels.
four pixels by four pixels or sixteen pixels by sixteen pixels. The assignment of colors to the pixels in the compressed
Block compression techniques seek to compress the non color data is realized by appropriately setting the corre
compressed color data that is provided for each of the pixels 20 sponding bits in a bit mask representing the cell. The bit
of a cell. The non-compressed color data of a pixel typically mask includes a bit at a specified position in the bit mask.
encodes red component information, green component The bit specifies the color of the pixel as one of the two
information and blue component information for the pixel. possible color choices. The average color of the set of
FIG. 1 shows a flowchart of the steps performed by one brighter pixels is associated with one of the possible bit
conventional block compression technique. In this tech 25 values (i.e., "1" or "0"), and the average color of the set of
nique, each frame of image data is divided into cells of four dimmer pixels is associated with the other possible bit value.
pixels by four pixels. The cells are compressed by an Hence, each bit value in the bit mask specifies the color of
iterative procedure until all of the cells in the image have the associated pixel as either the average color of the set of
been compressed. In accordance with this iterative proce 30 brighter pixels or the average color of the set of dimmer
dure, before compressing each cell in the image, the tech pixels.
nique checks whether compression is complete for the frame The above discussion has focused on the instances in
by determining if any cells remain to be compressed (step which the variance is less than the threshold value, as
10). If no cells remain to be compressed, the compression of checked by step 20 of FIG. 2. If, however, the variance is
the frame is complete. In contrast, if there are still cells left 35 greater than the threshold, the steps shown in FIG. 5 are
to be compressed, the color data for the next cell is obtained performed. In particular, the cell of four pixels by four pixels
(step 12), and the color data for the cell is compressed (step is divided into four sub-cells of two pixels by two pixels
14). (step 42). The block compression technique of FIG. 1 then
FIG. 2 is a flowchart showing the steps performed by the applies the compression approach of FIG. 3 iteratively for
block compression technique of FIG.1 in compressing a cell 40 each of the two-by-two sub-cells. Each sub-cell is treated as
of the image (see step 14 in FIG. 1). Initially, the luminance if it is a separate cell. In order to iteratively apply this
for each pixel in the cell is calculated (step 16). Once the approach, a check is made to determine whether any two
luminance for all of the pixels in the cells have been by-two sub-cells are left to be examined (step 44). The
calculated, the minimum luminance (min) and the maximum non-compressed color data for the next sub-cell of two
luminance (max) among the pixels of the cell are determined 45 pixels by two pixels is obtained (step 46), and the above
(step 18). The difference between the maximum luminance described steps of the flowchart of FIG. 3 are performed.
and the minimum luminance is then compared with a fixed, The main difference between the application of these steps
non-adjustable threshold value (step 20). The difference with the sub-cells and the application of these steps with the
between the maximum luminance and the minimum lumi four-pixels-by-four-pixels cells is the difference in the size
nance specifies the variance in the luminances of the pixels. 50 of the cells to which the steps are applied. In particular, each
This value of the variance is used to determine what com sub-cell is two pixels by two pixels, whereas each cell is four
pression approach to employ. If the variance is not greater pixels by four pixels. Steps 44 and 46 and the steps of FIG.
than the threshold, the steps shown in the flowchart of FIG. 3 are repeated for each of the remaining sub-cells until all
3 are performed. However, if the variance is greater than the four of the sub-cells have been compressed.
threshold, the steps shown in the flowchart of FIG. 5 are 55 The compressed color data resulting from the above
performed.
described compression technique is shown in FIGS. 6a and
The steps shown in the flowchart of FIG. 3 will now be 6b. FIG. 6a shows the resulting compressed color data for
described. The average luminance of the pixels in the cell is the first described approach, wherein each cell of four pixels
calculated (step 22). A check is made to determine whether by four pixels is compressed without further division into
any pixels are left to be examined in the cell (step 24). If an 60 sub-cells. The compressed color data includes a sixteen-bit
unexamined pixel remains, the pixel is examined by obtain bit mask 48 holding a bit for each pixel at a specified
ing the luminance of the pixel (step 26) and comparing the position in the bit mask. Each bit specifies the color of each
luminance of the pixel with the average luminance of the cell pixel in the cell as being one of two choices: the average
(step 28). If the luminance of the pixel is not greater than the color of the set of brighter pixels or the average color of the
average luminance, the pixel is marked as being in the set of 65 set of dimmer pixels. In particular, a '1' value for a bit in the
dimmer pixels (step 30). On the other hand, if the luminance bit mask specifies a first color of the two possible colors for
of the pixel is greater than the average luminance, the pixel the pixels, and a "0" value for the bit specifies the other
 5,544,286
3 4
possible color. The color information for the set of brighter In order to determine whether all of the pixels in the first
pixels is stored in a data word 50. The data word 50 includes cell have color data of substantially the same color, the
five bits dedicated to red component information, five bits following steps may be performed. First, an average color is
dedicated to green component information and five bits calculated for the pixels in the first cell. Color data for a third
dedicated to blue component information. The leading bit of cell is created. Each pixel in the third cell has color data that
the data word 50 has a value of "0". An analogous data word is equal to the calculated average color of the pixels in the
52 is produced for encoding the average color of the set of first cell. The color data of each pixel in the first cell is
dimmer pixels. compared with color data of a corresponding pixel in the
FIG. 6b shows the compressed color data, wherein the third cell to produce a total difference in color between
original cell is divided into four two-pixels-by-two-pixels 10 pixels of the first cell and the third cell. A determination is
made whether the total difference in color is less than a first
sub-cells. The compressed color data includes a sixteen-bit threshold. If the total difference is less than the first thresh
bit mask 54 that designates the colors of the respective pixels
in the cell. The assignment of pixels to sub-cells is fixed and, old, all the pixels in the first cell have color data of
thus, which sub-cell a pixel belongs to is known. Accord substantially the same color.
ingly, the two color choices for each pixel are known, and 15 A quality level may be set for the compressed color data.
the color for any pixel may be designated by a single bit in The first threshold, described above, may be calculated as a
the bit mask 54. The bit mask 54 designates these choices by value that is inversely proportional to the desired quality
including a “0” or '1' in the bit position for the associated level. When the desired quality level changes to a new
pixel. The compressed color information also includes, for quality level, the first threshold may be recalculated to a
each of the sub-cells, a data word encoding the average color 20 value that is inversely proportional to the new quality level.
of the set of brighter pixels and a data word encoding the The method may also include the step of comparing the
average color of the set of dimmer pixels. Thus, data words variance in color between the first cell and the third cell with
56 and 58 encode the color information for sub-cell iO. a second threshold. If the variance in color exceeds the
Similarly, data words 60 and 62 encode the color informa second threshold, the first cell is divided into sub-cells. Each
tion for sub-cell #1; data words 64 and 66 encode the color 25 sub-cell includes a subset of the pixels of the first cell. For
information for sub-cell #2; and data words 68 and 70 each sub-cell, the pixels of the sub-cell are divided into a set
encode the sub-cell information for cell #3. The leading bit of brighter pixels and a set of dimmer pixels. The average
of word 56, however, has a value of '1' rather than a value color of pixels in the set of brighter pixels is calculated, as
of '0' for the two-color case, so that it is known that the well as the average color of pixels in the set of dimmer
color information following the bit mask is for two-by-two 30 pixels. The compressed color data is produced by assigning
sub-cells rather than a four-by-four cell. the average color of the set of brighter pixels to each pixel
in the set of brighter pixels and assigning the average color
SUMMARY OF THE INVENTION of the set of dimmer pixels to each pixel in the set of dimmer
pixels.
35
It is an object of the present invention to provide an The second threshold may be selected as a value that is
improved digital video data compression technique. inversely proportional to a selected quality level. Hence, if
It is a further object of the present invention to provide a the quality level changes, the second threshold value is
digital video data compression technique that realizes recalculated.
greater compression than conventional techniques. 40 The compressed color data produced by this method may
The foregoing and other objects are realized by the assume different formats. In particular, the compressed color
present invention. In accordance with one aspect of the data may be in an eight-bit format or, alternatively, in a
present invention, a method of compressing color data for sixteen-bit format.
pixels in a first cell of a frame of color data to produce In accordance with another aspect of the present inven
compressed color data is practiced in a data processing 45 tion, a method of decompressing compressed color data for
system. In this method, the color data for each pixel in the pixels of a cell is performed in a data processing system. In
first cell is compared with color data for a corresponding this method, a determination is made whether the color data
pixel in a second cell of pixels in a previous frame of color for the cell is compressed as a solid-color cell. If the color
data. This comparison determines whether the color data in data for the cell is compressed as a solid-color cell, the color
the first cell and the color data in the second cell are 50 data of the cell is decompressed by assigning each pixel in
substantially the same. If the color data in the respective the cell a color that is encoded in the compressed color data.
cells is substantially the same, the compressed color data is A determination is also made whether the color data for the
produced by encoding the color data for the pixels of the first cell is compressed as a two-color cell. If the color data for
cell as a duplicate of the color data for the pixels of the the cell is compressed as a two-color cell, the color data of
second cell (i.e., encoding the cell as a "skip cell'). 55 the cell is decompressed by assigning each pixel one of two
After determining whether color data in the first cell is colors encoded in encodings of the compressed color data,
Substantially the same as the color data in the second cell, it based upon a corresponding value in a bit mask that is
may be determined whether all the pixels in the first cell contained in the compressed color data.
have color data of substantially the same color. If all of the This method may also include the additional steps of
pixels in the first cell have color data of substantially the 60 determining whether the cell is a skip cell and, if the cell is
same color, the compressed color data is produced by a skip cell, decompressing the color data of the cell by
encoding the first cell as a cell of the same color. The assigning each pixel in the cell a color that has been assigned
resulting compressed color data may include fixed bit values to a corresponding pixel in a second cell of a preceding
that specify that the pixels of the first cell are all substan frame. This method may further include the additional steps
tially the same color. The compressed color data may also 65 of determining whether the cell is an eight-color cell, and if
include an encoding of an average color of the pixels in the the cell is an eight-color cell, decompressing the color data
cell. of the cell by assigning each pixel in each sub-cell one of
 5,544,286
5 6
two colors encoded for the sub-cell in the compressed color FIG. 18b illustrates the sixteen-bit format produced by the
data by a corresponding value in a bit mask that is included preferred embodiment for the compressed color data for a
in the compressed color data. two-color cell.
FIG. 19 illustrates the sequencing of sub-cells used by the
BRIEF DESCRIPTION OF THE DRAWINGS preferred embodiment when eight-color cell compression is
used.
FIG. 1 is a flowchart illustrating the iterative approach
adopted by a prior art system for compressing digital video FIG.20a illustrates the eight-bit format compressed color
data. data produced by the preferred embodiment for an eight
color cell.
FIG. 2 is a flowchart illustrating the steps performed on 10
each cell to compress the cell in the conventional system of FIG. 20b illustrates the sixteen-bit format compressed
FIG. 1. color data produced by the preferred embodiment for an
eight-color cell.
FIG. 3 is a flowchart illustrating the steps performed by
the conventional system of FIG. 1, when the variance in FIG. 21a is a flowchart illustrating the steps performed in
luminances in the cell is less than a threshold value. 15 decompressing eight-bit format compressed color data in
FIG. 4 is a flowchart illustrating the steps that are per accordance with another aspect of the preferred embodiment
described herein.
formed to produce the compressed color data in the flow FIG.21b is a flowchart illustrating the steps performed in
chart of FIG. 3.
FIG. 5 is a flowchart illustrating the steps performed by decompressing compressed color data in a sixteen-bit format
the conventional system of FIG. 1, when the variance in
20 in accordance with another aspect of the preferred embodi
ment described herein.
luminances in the cell exceeds a threshold value.
FIG. 6a illustrates the format of the compressed color data DETAILED DESCRIPTION OF THE
resulting from execution of the steps shown in FIG. 3. INVENTION
FIG. 6b shows the format of the compressed color data 25
The present invention provides greater compression of
resulting from execution of the steps shown in FIG. 5. digital video data than the conventional system shown in
FIG. 7 illustrates an example data processing system for FIGS. 1 through 6b. Greater compression is realized by
practicing a preferred embodiment of the present invention using techniques for compressing a cell having color data of
described herein.
30 substantially one color and/or by using techniques for com
FIG. 8 illustrates the sequencing of cells used by the pressing a cell having color data that is substantially iden
preferred embodiment of the invention described herein. tical to the color data of at least one previous cell. In
FIG. 9 illustrates the sequencing of pixels in a cell used addition, the preferred embodiment of the present invention
by the preferred embodiment described herein. described herein may provide for two-color compression of
FIG. 10a is a flowchart illustrating the steps performed to 35 a cell and eight-color compression of a cell.
calculate thresholds for the preferred embodiment described The present invention may be used in applications that
herein. require compression of digital video data. A preferred
FIG. 10b is a flowchart illustrating the steps performed embodiment described herein provides both compression
when the quality level is changed to a new level in the and decompression of digital video data. The preferred
preferred embodiment described herein. 40 embodiment described herein may be encoded in software
FIG. 11 is a flowchart illustrating the steps performed to that is executed on any one of a number of different data
select the output format for the compressed color data in the processing configurations, including the data processing
preferred embodiment described herein. system 72 shown in FIG. 7. In this data processing system
FIG. 12 is a flowchart illustrating an overview of the steps 72, a central processing unit (CPU) 74 executes instructions
performed by the preferred embodiment described herein. 45 of programs held in memory 76. The CPU 74 communicates
with the memory 76 over a communication path 82, such as
FIG. 13 is a flowchart illustrating the steps performed by a bus, interconnection network or other suitable communi
the preferred embodiment to determine if a cell is a skip cell. cation mechanism. The CPU 74 may also communicate with
FIG. 14 illustrates the format produced by the first pre a video adapter 80 through the communication path 82. The
ferred embodiment for the compressed color data for a skip 50 video adapter serves as an interface between the CPU 74 and
cell. a video display device 78. When the CPU 74 desires to
FIG. 15 is a flowchart illustrating the steps performed by display an image on the video display device 78, it passes
the preferred embodiment described herein to determine if a appropriate digital video data to the video adapter 80. The
cell is a solid-color cell. video adapter 80 generates electrical signals that are passed
FIG. 16a illustrates the eight-bit output format produced 55 to the video display device 78 to cause the image to be
by the preferred embodiment for the compressed color data displayed on the video display device.
for a solid-color cell. In accordance with the preferred embodiment described
FIG. 16b illustrates the sixteen-bit format produced by the herein, a frame of video data is divided into cells of
preferred embodiment for the compressed color data for a four-pixels-by-four-pixels, as shown in FIG.8. Those skilled
Solid-color cell. in the art will appreciate that this cell size is merely
FIG. 17 is a flowchart illustrating the steps performed by illustrative and that other cell sizes may be used. Each row
the preferred embodiment to determine if a cell is a two of a frame of the image includes "W" cells (see FIG. 8),
color cell. where "W" specifies the width of a row in number of cells.
Each column includes “H” cells, where "H" specifies the
FIG. 18a illustrates the eight-bit format produced by the 65 height of a column in number of cells. Each successive cell
preferred embodiment for the compressed color data for a is assigned a position in a sequence of data for the frame, as
two-color cell. shown in FIG.8. The sequence of cells in the frame begins
 5,544,286
7 8
with cell #0 in the lower left-hand corner of the frame and identical to a corresponding cell in a previous frame. If the
progresses across the row and up successive rows of cells, cell is a skip cell, the cell is compressed accordingly (step
as shown in FIG. 8. (Note that each successive cell is given 104). If the cell is not a skip cell, a determination is made
a successively higher number.) The compressed data for the whether the cell is a solid-color cell (step 106). A solid-color
cells is stored according to this sequence ranging from cell cell is one in which the pixels in the cell are substantially
() to cell WH-1. identical, and therefore, the cell can be encoded as a single
As shown in FIG. 9, the color data of pixels within each color cell. If the cell is a solid-color cell, the cell is
cell are also stored according to a sequence. The color data compressed as a solid-color cell (step 108). The data pro
of the pixels follow a sequence beginning with pixel #0 in cessing system 72 (FIG. 7) then checks whether the cell is
the lower left-hand corner of each cell. The sequence 10 a two-color cell (step 110). If the cell is a two-color cell, the
progresses across each pixel row of the cell and wraps cell is compressed as a two-color cell (step 112). If the cell
around to the beginning of the next pixel row for each is not a solid-color cell, a skip cell or a two-color cell, the
successive pixel row up to pixel "F". cell is encoded as an eight-color cell (step 114). The above
In accordance with one aspect of the preferred embodi described steps are repeated until all the cells of the frame
ment described herein, the cells of a frame of digital video 15 have been properly compressed.
data are compressed in accordance with the steps shown in In order to more fully understand the iterative technique
the flowchart of FIG. 12. Before discussing the details of the shown in the flowchart of FIG. 12, it is helpful to further
flowchart of FIG. 12, it is helpful to consider some prelimi examine the details of determining cell type in steps 102,
nary matters. 106 and 110 and to further examine the details of encoding
The data processing system 72 uses a number of thresh 20 cells in steps 104,108 and 112.
olds for decision making during the compression of the cells. To make the determination of whether the cell is a skip
These thresholds are calculated in accordance with the steps cell (see step 102 in FIG. 12), the data processing system 72
shown in the flowchart of FIG.10a. Initially, a user specifies (FIG. 7) performs the steps shown in the flowchart of FIG.
a quality value (step 84), such as a value in the range from 13. First, a cell is obtained from a previous frame, such as
0 to 10,000, wherein 10,000 represents the highest quality 25 the immediately succeeding frame (step 116). Typically, this
level and 0 represents the lowest quality level. During cell is a like positioned cell in the previous frame as the cell
compression in this preferred embodiment, certain data is being compressed. As the color data for the cell from the
typically lost in the compression (i.e., it is a lossy compres previous frame may already be compressed, it may be
sion technique). The user-selected quality level specifies the necessary to expand the color data (step 118 in FIG. 13). The
acceptable level of data loss. This quality level is then used 30 color difference between the current cell and the cell from
to calculate a base threshold value (step 86). The base the previous frame is then calculated, using the sum of the
threshold value is calculated as: squares of the differences approach (step 120). The color
difference may be calculated as the sum of the squares of the
(1-(quality level)/10,000)*((256)/2) differences between the red, green and blue components of
35 the respective pixels (i.e., the Euclidean distance between
Those skilled in the art will appreciate that the thresholds the red, green and blue components of the respective cells in
may also be calculated in other ways. The described a red, green, blue color space). A determination is then made
approach is merely illustrative. whether the color difference is greater than the skip-cell
A solid-color threshold and a skip-cell threshold are threshold (step 122). If the color difference is not greater
calculated from this base threshold as fixed ratios of the base than the skip-cell threshold, the cell is designated as a skip
threshold (steps 88 and 90). The ratios may be varied based cell (step 124). In contrast, if the color difference is greater
on the nature of the application and/or type of image data than the skip-cell threshold, the difference between the two
being compressed. A user of the data processing system 72 cells is too large, and the cell is designated as not being a
(FIG. 7) may subsequently change the quality level to a new skip cell (step 126).
level (step 89 in FIG. 10b). New thresholds are then calcu 45 The designation as a skip cell or as not a skip cell is used
lated using the new quality level (step 91). to make the decision in step 102 of FIG. 12. If it is
The preferred embodiment described herein has the capa determined that the cell is a skip cell, the cell is encoded as
bility of producing either an eight-bit output format or a a skip cell (step 104 in FIG. 12). The format for the
sixteen-bit output format. Thus, the data processing system compressed encoded skip cell is shown in FIG. 14. The
72 (FIG. 7) must determine which format a user requires 50 format is the same for both a sixteen-bit format and an
(step 92 in FIG. 11). If the user requires a sixteen-bit format, eight-bit format. The compressed output is a single data
code for compressing the digital video data into a sixteen-bit word 128 having a fixed value of “100001' for bits fifteen
output format is executed (step 96). In contrast, if the user through ten. The remaining bits specify the number of cells
requires an eight-bit output format, code for compressing the that are substantially identical.
digital video data compressed into an eight-bit output format 55 In order to determine whether a cell is a solid-color cell
is executed (step 94). (see step 106 in FIG. 12), the data processing system 72
As mentioned above, the cells of a frame of digital video (FIG. 7) performs the steps shown in the flowchart of FIG.
data are compressed by executing the steps shown in the 15. First, the average color of the pixels in the cell is
flowchart of FIG. 12. The data processing system 72 (FIG. calculated (step 130). An imaginary cell is then built in
7) checks whether any cells to be compressed remain in the 60 which all of the pixels have the average color (step 132). The
frame (step 98 in FIG. 12). If there are no cells left to be total color difference between the cell and the imaginary cell
compressed, the entire frame has been fully compressed. is then calculated (step 134). The calculated total color
However, if there are cells left to be compressed, the system difference is then compared with the solid-color threshold to
gets the next one of the remaining cells to be compressed determine if the total color difference is greater than the
(step 100). The data processing system 72 (FIG. 7) then 65 solid-color threshold (step 136). If the total color difference
checks whether the cell is a skip cell (step 102 in FIG. 12). is not greater than the solid-color threshold, there is an
A skip cell is a cell having color data that is substantially insubstantial difference in colors of pixels in the two cells
 5,544,286
9 10
and the cell is designated as a solid-color cell (step 138). On components. Bit fifteen of the encoding 162 is set as "1".
the other hand, if the total color difference is greater than the The encoding 164 of the color for "0" in the bit mask,
solid-color threshold, the cell is designated as not a solid likewise, includes red component, green component and
color cell (step 140). blue component values. The encoding 164 differs from the
Using the designation provided in either step 138 or step encoding 162 in that bit fifteen is set to have a value of "0".
140 of FIG. 15, the data processing system 72 (FIG. 7) The encoding of the cell as an eight-color cell (see step
determines whether to encode the cell as a solid-color cell 114 in FIG. 12) is performed by executing the steps shown
(step 106 of FIG. 12). If it is determined that the cell should in FIG. 5. These steps divide the cell into four two-by-two
be encoded as a solid-color cell, the cell may be encoded in sub-cells, as shown in FIG. 19. The sub-cells are then
either an eight-bit format or a sixteen-bit format. FIG. 16a 10 separately compressed as two-color cells by performing the
shows an encoded data word 142 for a solid-color cell. The steps shown in FIG. 5. When an eight-bit format for the
eight most significant bits are set to have a value of output is selected, the resulting compressed color data
"10000000". The remaining eight least significant bits hold assumes a format like that shown in FIG.20a. In particular,
a binary color code value. If, however, a sixteen-bit output a normalized sixteen-bit bit mask 166 is produced, wherein
format is desired, the format of FIG. 16b is produced. The 15 a "l' is stored in bit positions fifteen and thirteen. For each
compressed output in the sixteen-bit output format is a data respective sub-cell, an eight-bit encoding of the color for “1”
word 144 having a five-bit red component, a five-bit green values in the bit mask and an eight-bit encoding of the color
component and a five-bit blue component. The most signifi for "0" values in the bit mask follows. Specifically, the
cant bit (i.e., bit fifteen) of the data word 144 has a value of encoding 168 encodes the color for “1” values for sub-cell
c. 1''. 20 #0; the encoding 170 encodes the color for "O' values for
The determination of whether a cell is a two-color cell sub-cell #0; the encoding 172 encodes the color for “1”
(see step 110 in FIG. 12) is performed by executing the steps values for sub-cell #1; the encoding 174 encodes the color
shown in the flowchart of FIG. 17. The average color of the for "0" values for sub-cell #1; the encoding 176 encodes the
pixels in the cell is calculated and a cell is built having pixels color for "1" values for sub-cell #2; the encoding 178
which all have the average color (step 146). The color 25 encodes the color for "O' values for sub-cell #2; the encod
difference between the cell and the built cell is calculated as ing 180 encodes the color for "1" values for sub-cell #3; and
the sum of the squares of the differences (step 148). The the encoding 182 encodes the color for "0" values for
color difference is then compared with the base threshold sub-cell #3.
(step 150). If the color difference is not greater than the base When a sixteen-bit format is selected and the cell is an
threshold, the cell is designated a two-color cell (step 152). 30 eight-color cell, the compressed color data is similar to that
If the color difference is greater than the base threshold, the shown in FIG.20a, but the colors are encoded in a sixteen
variance of pixel color in the cell is large enough that bit format rather than an eight-bit format. Thus, as shown in
two-color compression will yield an unsatisfactory result FIG. 20b, sixteen-bit encodings 186, 188, 190, 192, 194,
and, thus, step 114 of FIG. 12 is performed to encode the cell 196, 198 and 200 encode the colors for the respective cells.
as an eight-color cell. The approach adopted by the preferred 35 The first bit (i.e., bit fifteen) of each of the encodings is set
embodiment herein for two-color compression differs from as "0". A sixteen-bit bit mask 184 is provided to specify
the two-color cell compression, described in the Background which colors of the pixels is assigned.
of the Invention, in that the base threshold is adjustable In accordance with another aspect of the preferred
depending upon the quality selected by the user. In the embodiment of the present invention described herein, data
conventional approach, the threshold is fixed and not readily 40 may be decompressed from the eight-bit compressed format
adjustable. or the sixteen-bit compressed format that is produced by the
In order to encode the cell as a two-color cell (see step 112 above-described compression technique. The steps per
in FIG. 12), the steps shown in FIGS.3 and 4 are performed, formed in decompressing digital video data in an eight-bit
as described above in the Background of the Invention. The format are shown in the flowchart of FIG. 21a. The leading
resulting compressed color data may be encoded in an 45 bits of a first word of information in a block of compressed
eight-bit format or a sixteen-bit format. FIG. 18a shows the color data specify how the cell was compressed. Once it is
compressed color data when an eight-bit format is used. The known how the cell was compressed, the cell may be
resulting compressed color data includes three parts: a bit decompressed accordingly. Decompression begins by exam
mask 154; an eight-bit encoding 156 of the color for "1" ining the first bit in the first word of a block of compressed
values in the bit mask; and an eight-bit encoding 158 of the 50 color data (step 202) If the first bit equals “0” the cell has
color for "0" values in the bit mask. The most significant bit been compressed as a two-color cell. Therefore, the cell is
(i.e., bit fifteen) of the bitmap 154 is normalized to always decompressed as a two-color cell (step 204) by assigning the
have a constant value of "0". Specifically, normalization color encoded in encoding 156 (FIG. 18a) for “1” values in
includes swapping the choice of color for "0" values and the the bit mask 154 to each pixel with “1” in the bit mask 154.
choice of color for '1' values if bit fifteen would, otherwise, 55 Analogously, each bit with a value of "0" in the bit mask 154
have a value of “1”. The eight-bit encodings 156 and 158 of is assigned the encoded color for "O' value that is encoded
colors each specify an index into a color table for a specific by encoding 158.
color. Next, in decompressing data in the eight-bit format, a
FIG. 18b shows the compressed color data for the sixteen determination is made whether the third bit in the first word
bit output format. In this instance, the compressed color data 60 of the block of compressed color data equals “1” (step 206
includes three parts: a bit mask 160; a sixteen-bit encoding in FIG. 21a). From step 202, it is already known that the first
162 of the color for '1' values in the bit mask; and a bit equals “1”. If both the third bit and the first bit equal “1”
sixteen-bit encoding 164 of the color for "0" values in the bit the cell has been compressed as an eight-color cell. The cell
mask. Bit fifteen of the bit mask 160 has a value of "0'. The is, hence, decompressed as an eight-color cell (step 210).
bit mask 160 is normalized, as described above, to ensure 65 Specifically, the pixels in sub-cell #0 (see FIG. 19) are
this "0" value forbit fifteen. The sixteen-bit encoding 162 of assigned either the color for “1” that is encoded by encoding
the color for "1" in the bit mask includes red, green and blue 168 (FIG. 20a) or the color for "0" that is encoded by
 5,544,286
11 12
encoding 170, based upon the value of the corresponding c) if the compressed color data for the first cell has not
bits in the bit mask 166. Similarly, the pixels in sub-cells #1, been produced by the preceding steps, producing the
#2 and #3 (see FIG. 19) are assigned the colors encoded by compressed color data by performing the following
encodings 172,174, 176, 178, 180 and 182, as specified by steps:
the bits in the bit mask 166 (FIG. 20a). (i) calculating the luminance of each of the pixels in the
If the first bit in the data block is a '1' and the third bit first cell;
is "0" a determination is made whether the first six bits equal (ii) dividing the pixels in the first cell into a set of
“100001” (step 208 in FIG. 21a). If the first six bits of the brighter pixels and a set of dimmer pixels, based on
first word of a data block equal "100001" the cell is a skip the calculated luminances;
cell The compressed data word is decompressed (step 212) 10 (iii) calculating an average color of pixels in the set of
by generating decompressed color data for the number of brighter pixels and calculating an average color of
"skipped cells' that is encoded in the compressed color data pixels in the set of dimmer pixels;
(see data word 128 of FIG. 14). The decompressed color (iv) assigning the average color of pixels in the set of
data is set to be equal to that of the cell in the previous frame brighter pixels to each of the pixels in the set of
that corresponds with the skip-cell compressed color data. brighter pixels; and
5
Lastly, if none of the other tests (i.e., steps 202, 206 and (v) assigning the average color of pixels in the set of
208) have been met, the cell must be compressed as a dimmer pixels to each of the pixels in the dimmer set
solid-color cell. Hence, the cell is decompressed as a solid of pixels.
color cell (step 214). In decompressing the cell as a solid 2. A method as recited in claim 1, further comprising the
color cell, each cell is assigned the average color encoded in steps of:
20 after producing the compressed color data by encoding
encoding 142 (FIG. 16a) the compressed color data.
FIG. 21b shows a flowchart of the steps performed when the color data for the pixels of the first cell as a
decompressing digital video data that has been compressed duplicate of the color data for the pixels of the second
in a sixteen-bit format. In step 218, a determination is made cell, determining whether all of the pixels in the first
whether the first six bits of the first word of the data block cell have color data of substantially the same color;
25 if all of the pixels in the first cell have color data of
equal “100001". If the first six bits equal this value, the cell substantially the same color, producing the compressed
is decompressed as a skip cell (step 220). Decompression of color data by encoding the first cell as a cell of the same
a skip cell is performed in a fashion that is analogous to the color pixels; and
decompression in the eight-bit format that is described
above. In accordance with this decompression approach, a 30
if all of the pixels in the first cell do not have color data
determination is next made whether the first bit has a value of substantially the same color, producing the com
of “1” (step 222). If the first bit has a value of “1” the cell pressed color data by performing the following steps:
is decompressed as a solid-color cell (step 224). The decom (i) calculating the luminance of each of the pixels in the
pression is analogous to the decompression performed with first cell;
compressed color data in an eight-bit format, except that the 35
(ii) dividing the pixels in the first cell into a set of
compressed color data is in a sixteen-bit format. brighter pixels and a set of dimmer pixels, based on
In step 226, it is checked whether the most significant bit the calculated luminances;
of the first color word following the bitmap in the com (iii) calculating an average color of pixels in the set of
pressed color data has a value of "1". If the most significant brighter pixels and calculating an average color of
bit of the first color word has a value of '1', the cell is 40
pixels in the set of dimmer pixels;
decompressed as a two-color cell in a fashion analogous to (iv) assigning the average color of pixels in the set of
the approach described above for compressed color data in brighter pixels to each of the pixels in the set of
an eight-bit format (step 228). Lastly, if the most significant brighter pixels; and
bit of the first color word is a '0' instead of a "1", the cell (v) assigning the average color of pixels in the set of
is decompressed as an eight-color cell in a fashion analogous dimmer pixels to each of the pixels in the dimmer set
to the above-described approach for an eight-bit format (step
45 of pixels.
230). 3. A method as recited in claim 2, wherein the step of
While the present invention has been described with producing the compressed color data by encoding the first
reference to a preferred embodiment thereof, those skilled in cell as a cell of the same color pixels further comprises the
the art will appreciate that various changes in form and step of:
50
scope may be made without departing from the spirit of the producing the compressed color data such that the com
present invention as defined in the appended claims. pressed color data includes fixed bit values that specify
I claim: that the first cell is a cell of the same color pixels and
1. In a data processing system, a method of compressing further includes an encoding of the average color of the
color data for pixels in a first cell of a frame of color data to 55
pixels in the first cell.
produce compressed color data, comprising the steps of: 4. A method as recited in claim 2, wherein the step of
a) for each pixel in the first cell, comparing the color data determining whether all of the pixels in the first cell have
for the pixel in the first cell with color data for a pixel color data of substantially the same color further comprises
in a second cell of pixels in a previous frame of color the steps of:
data to determine whether the color data in the first cell 60 calculating an average color of the pixels of the first cell;
and the color data in the second cell are substantially creating color data for a third cell in which each pixel has
the same; color data equal to the calculated average color of the
b) if the color data in the first cell is substantially the same pixels in the first cell;
as the color data for the second cell, producing the comparing the color data of each pixel in the first cell with
compressed color data by encoding the color data for 65 the color data of a corresponding pixel in the third cell
the pixels of the first cell as a duplicate of the color data to produce a total difference in color between pixels of
for the pixels of the second cell; the first cell and the third cell;
 5,544,286
13 14
determining whether the total difference is less than a first cell and the second cell, and using the variance in color
threshold; and between the first cell and the second cell to determine
if the total difference is less than the first threshold, whether the color data in the first cell and the color data
determining that all of the pixels in the first cell have in the second cell are substantially the same;
color data of substantially the same color. 5 b) if the color data in the first cell is substantially the same
5. A method as recited in claim 4, further comprising the as the color data in the second cell, producing the
steps of: compressed color data by encoding the color data for
setting a desired quality level for the compressed color the pixels of the first cell as a duplicate of the color data
data, and for the pixels of the second cell;
calculating the first threshold as a value that is inversely 10 c) if the color data in the first cell is not substantially the.
proportional to the desired quality level. same as the color data in the second cell:
6. A method as recited in claim 5, further comprising the (i) calculating an average color of pixels in the first cell;
steps of: (ii) creating color data for a third cell in which each
changing the desired quality level to a new quality level; pixel has color data equal to the calculated average
and 15 color of pixels in the first cell;
(iii) comparing the color data of each pixel in the first
recalculating the first threshold as a value that is inversely cell with color data of a corresponding pixel in the
proportional to the new quality level. third cell to produce a variance in color between the
7. A method as recited claim 1, further comprising the first cell and the third cell;
steps of: 20 (iv) based on the variance in color between the first cell
after step b): and the third cell, determining whether all of the
(i) calculating an average color of pixels in the first cell; pixels in the first cell have color data of substantially
(ii) creating color data for a third cell in which each the same color;
pixel has color data equal to the calculated average d) if all of the pixels in the first cell have color data of
color of pixels in the first cell; substantially the same color, producing the compressed
(iii) comparing the color data of each pixel in the first color data by encoding the first cell as a cell of the same
cell with color data of a corresponding pixel in the color;
third cell to produce a variance in color between the e) calculating luminances of pixels in the first cell;
first cell and the third cell; f) if the variance in color between the first cell and the
(iv) if the variance in color between the first cell and the 30 third cell is less than a first threshold, producing the
third cell exceeds a second threshold: compressed color data by encoding color data for each
dividing the first cell into sub-cells, each sub-cell of the pixels in the first cell as having one of two colors;
comprising a subset of the pixels of the first cell; and
for each sub-cell, dividing the pixels of the sub-cell g) if the variance in color between the first cell and the
into a set of brighter pixels and a set of dimmer 35 third cell is greater than the first threshold:
pixels; (i) dividing the first cell into sub-cells of pixels,
calculating an average color of pixels in the set of wherein each sub-cell is a subset of the first cell; and
brighter pixels and calculating an average color of (ii) for each sub-cell, producing the compressed color
pixels in the set of dimmer pixels; and data by encoding each pixel as one of two colors,
producing the compressed color data by assigning whereby the two colors used in encoding pixels
the average color of the set of brighter pixels to differ between at least two of the sub-cells.
each pixel in the set of brighter pixels and assign 13. A method as recited in claim 12, wherein the step of
ing the average color of the set of dimmer pixels producing the compressed color data by encoding the first
to each pixel in the set of dimmer pixels. cell as a cell of the same color further comprises the step of:
8. A method as recited in claim 7, further comprising the 45 producing the compressed color data such that the com
steps of:
selecting a desired quality level for compression; and
pressed color data includes fixed bit values that specify
that the pixels of the first cell are all substantially the
calculating the second threshold as a value that is same color and further includes an encoding of the
inversely proportional to the desired quality level. average color of the pixels in the first cell.
9. A method as recited in claim 8, further comprising the 50 14. A method as recited in claim 13, wherein the step of
steps of: determining whether all of the pixels in the first cell have
changing the desired quality level to a new quality level; color data of substantially the same color further comprises
and the steps of:
recalculating the second threshold as a value that is 55 determining whether the variance in color between the
inversely proportional to the new quality level. first cell and the third cell is less than a second
10. A method as recited in claim 1, wherein the com threshold; and
pressed color data is in an eight-bit format. if the variance in color between the first cell and the third
11. A method as recited in claim 1, wherein the com cell is less than the second threshold, determining that
pressed color data is in a sixteen-bit format. all of the pixels in the first cell have color data of
12. In a data processing system, a method of compressing substantially the same color.
color data for pixels in a first cell of a frame of color data to 15. A method as recited in claim 12, further comprising
produce compressed color data, comprising the steps of: the steps of:
a) for each pixel in the first cell, comparing the color data setting a desired quality level for the compressed color
for the pixel in the first cell with color data for a pixel 65 data; and
in a second cell of pixels in a previous frame of color calculating the first threshold as a value that is inversely
data to determine a variance in color between the first proportional to the desired quality level.
 5,544,286
15 16
16. A method as recited in claim 15, further comprising assigning the average colors of pixels in the set of dimmer
the steps of: pixels to each of the pixels in the set of dimmer pixels.
changing the desired quality level to a new quality level; 20. A method as recited in claim 12, wherein the step of,
and
recalculating the first threshold as a value that is inversely 5 for each sub-cell, producing the compressed color data by
proportional to the new quality level. encoding each pixel as the two colors further comprises the
17. A method as recited in claim 12, wherein the com steps of:
pressed color data is in an eight-bit format. dividing the pixels of the sub-cell into a set of brighter
18. A method as recited in claim 12, wherein the com
pressed color data is in a sixteen-bit format. O pixels and a set of dimmer pixels, based on the calcu
19. A method as recited in claim 12, wherein the step of lated luminances;
producing the compressed color data by encoding color data calculating an average color of pixels in the set of brighter
for each of the pixels as having one color selected from a
group of two colors, further comprises the steps of: pixels and calculating an average color of pixels in the
dividing the pixels in the first cell into a set of brighter set of dimmer pixels; and
pixels and a set of dimmer pixels based on the calcu producing the compressed color data by assigning the
lated luminances; average color of the set of brighter pixels to each pixel
calculating an average color of pixels in the set of brighter in the set of brighter pixels and assigning the average
pixels and calculating an average color of pixels in the
set of dimmer pixels; 20 color of the set of dimmer pixels to each pixel in the set
assigning the average color of pixels in the set of brighter of dimmer pixels.
pixels to each of the pixels in the set of brighter pixels;
and