Hindawi Publishing Corporation

EURASIP Journal on Advances in Signal Processing

Volume 2007, Article ID 93027, 15 pages
doi:10.1155/2007/93027

Research Article
3D Game Content Distributed Adaptation in
Heterogeneous Environments

Francisco Morán,1 Marius Preda,2 Gauthier Lafruit,3 Paulo Villegas,4 and Robert-Paul Berretty5
1 Grupo de Tratamiento de Imágenes, Universidad Politécnica de Madrid, 28040 Madrid, Spain
2 Département ARTEMIS, Institut National des Télécommunications, 91011 Évry, France
3 DESICS, Interuniversitair Micro Electronica Centrum, 3001 Leuven, Belgium
4 Telefónica Investigación y Desarrollo, 47151 Boecillo, Spain
5 Philips Research, 5656 AE Eindhoven, The Netherlands

Received 31 August 2006; Revised 9 January 2007; Accepted 5 July 2007

Recommended by Yap-Peng Tan

Most current multiplayer 3D games can only be played on a single dedicated platform (a particular computer, console, or cell
phone), requiring specifically designed content and communication over a predefined network. Below we show how, by using
signal processing techniques such as multiresolution representation and scalable coding for all the components of a 3D graphics
object (geometry, texture, and animation), we enable online dynamic content adaptation, and thus delivery of the same content
over heterogeneous networks to terminals with very different profiles, and its rendering on them. We present quantitative results
demonstrating how the best displayed quality versus computational complexity versus bandwidth tradeoffs have been achieved,
given the distributed resources available over the end-to-end content delivery chain. Additionally, we use state-of-the-art, stan-
dardised content representation and compression formats (MPEG-4 AFX, JPEG 2000, XML), enabling deployment over existing
infrastructure, while keeping hooks to well-established practices in the game industry.

Copyright © 2007 Francisco Morán et al. This is an open access article distributed under the Creative Commons Attribution
License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly
cited.

1. INTRODUCTION (i) game developers must tailor content to specific termi-

nal/network combinations set as a priori targets, and
OLGA (www.ist-olga.org) is the short name of a Specific cannot provide the adequate quality for a new such
Targeted REsearch Project (STREP) partially funded, from combination without substantial effort;
April 2004 to September 2006, by the European Com- (ii) terminal builders and network operators want to di-
mission under the Information Society Technologies pri- versify their platform characteristics while still being
ority of its Sixth Framework Programme. Its full name able to allow for game playing;
comes from its aim: to develop “a unified scalable frame- (iii) most of all, end-users want to roam attractive games in
work for On-Line GAming.” The core of the research car- different usage contexts, inside and outside the home,
ried within OLGA, and presented in this paper, was on without being trapped into a single terminal/network
the challenging topic of 3D game content distributed adap- configuration.
tation in heterogeneous environments. Multiplatform on- To overcome platform-centricness and move towards user-
line gaming is an excellent scenario to prove the poten- centricness, we decided to develop a role-playing game
tial benefits of 4D (animated 3D) content adaptation in (RPG) [1] named GOAL that would yield similar user ex-
the current multimedia world, which still suffers from periences on MS Windows-based personal computers (PCs)
“platform-centricness” and craves for “user-centricness” and on cell phones (CPs) running Symbian OS. Thanks to
and interoperability, despite technological progress over the OLGA framework, it would be—and in fact is—possible
recent years. Indeed, the terminal and network hetero- to automatically adapt and render the same textured 4D con-
geneity characterising online games make them the per- tent at wildly different qualities and frame rates, according
fect example of a completely platform-centric multimedia to each particular network/terminal profile, as suggested by
application: Figure 1.
2 EURASIP Journal on Advances in Signal Processing

content adaptation for heterogeneous environments (plat-

forms/terminals and networks) based on international stan-
dards. The subsections 2.1 to 2.3 below describe how, for
each kind of data (3D geometry, 2D textures, animation),
different tradeoffs can be achieved between the quality of the
decoded content versus the required memory footprint and
execution time (which are clearly platform-related aspects)
versus the bit-rate (which is mostly a network-related one).

2.1. 3D geometry data

Our research in the field of multiresolution coding of static

3D shapes targeted methods more suitable for resource-
limited devices than the “WaveSurf ” tool in MPEG-4 AFX
[2], based on modelling first a given 3D shape (e.g., an ar-
Figure 1: Screen shots from both the PC and CP versions of GOAL, bitrary connectivity mesh) as a wavelet subdivision surface
OLGA’s game. (WSS), and then coding it thanks to the set partitioning in
hierarchical trees (SPIHT) technique [5].
Two different types of scalability can be sought in 3D
geometry coding, as in the case of image coding: signal to
We also decided to segregate, from the necessary game noise ratio (SNR) scalability gives the possibility of decod-
logic network, a novel content delivery network which would ing a 3D model (or image) with different degrees of fidelity
enable our framework to automatically adapt content in a (reconstruction error), whereas spatial scalability allows to
distributed way. Furthermore, this distributed content deliv- decode it with different spatial resolutions, that is, number
ery network would let players publish their own content and, of vertices/facets (or pixels). For a decade already, SPIHT has
to this end, we created content authoring tools meant to be been the reference for other scalable coding techniques based
used not only by game designers but also by end users. The on the wavelet transform. It was originally designed to code
delivery of content over different networks and its distributed scalar wavelet coefficients, but has been extended to handle
adaptation imposed three basic requirements: 3D coefficients, such as the ones resulting from RGB images
(i) the volume of data required by the geometry, textures, or 3D surfaces modelled thanks to WSSs [6–8].
and animation of 4D models is usually huge, so some WSSs are a powerful multiresolution representation
form of content compression was a must; paradigm for 3D shapes but the problem of SPIHT is that, al-
(ii) to ensure interoperability, de jure international stan- though its bit-streams are SNR scalable, they are not spatially
dards such as MPEG-4 Animated Framework eXten- scalable. SPIHT bit-streams cannot be easily parsed accord-
sion (AFX) [2], JPEG 2000 [3], and XML [4] would be ing to a given maximum resolution (i.e., number of pixels or
used, and improved if possible; triangles) or level of detail (LOD, i.e., generation of the sub-
(iii) as content adaptation processes were to run on the division process) tolerated by the decoder, and there is little
same PCs of some of the players, the processing load point in encoding a 3D mesh with thousands of triangles if
needed by the content adaptation tasks had to be kept the CP that must render it can barely handle hundreds, and
to a minimum. even less if, anyway, nobody will be able to tell the difference
between a 100-triangle mesh and a 1000-triangle one when
Section 2 elaborates on the core of our research: how scal- rendered on a screen of 200 × 200 pixels! Furthermore, from
able coding can be exploited to adapt the different kinds of the memory viewpoint (as opposed to the rendering one),
content data (3D geometry, textures, and animation) in a having an SNR scalable bit-stream that may have bits corre-
specific terminal, while achieving excellent quality versus bit- sponding to details of LOD 3 before those of LOD 1 makes
rate versus memory versus execution time tradeoffs. also little sense, as the decoding process alone will exhaust all
The rest of our research results highlighted in Sections 3 the CP resources, even if memory is not allocated for the tri-
and 4; the former explains how separating the game logic and angles of LOD 3 (which will never be rendered), their detail
content delivery networks allows for a high degree of scalabil- trees must be created to follow the SPIHT algorithm.
ity against the number of clients, and the latter gives some de- The outcome of our research is the progressive lower
tails on the 3D rendering on heterogeneous platforms, stress- trees of wavelets (PLTW) technique [9], whose main nov-
ing our achievements related to auto-stereoscopic displays. elty is that the resulting bit-stream does not impose on
Finally, Section 5 concludes our presentation. the less powerful decoders the need of building unneces-
sary detail trees. With PLTW, the set of wavelet coefficients
2. CROSS-PLATFORM AND CROSS-NETWORK 4D is also hierarchically traversed, but coded on a per-LOD ba-
CONTENT ADAPTATION sis, thus yielding a bit-stream with “local SNR scalability”
and, at the same time, “global spatial scalability.” The de-
This section presents the core of the research carried out coder first receives all the coefficients corresponding to an
within the OLGA project, which focussed on textured 4D LOD and, only when it has finished reading them, it proceeds
Francisco Morán et al. 3

Table 1: Progressive reconstruction of the bunny model from a PLTW bit-stream.

Bit-stream
LOD 0: LOD 1: LOD 2: LOD 3:
read
base mesh one subdivision two subdivisions three subdivisions
per LOD

0%

50%

100%

(if it has enough resources) with those from the next. Never- pression shown in one curve), and the “WaveSurf ” tool of
theless, thanks to bit-plane encoding, the first received bits MPEG-4, which also uses SPIHT, but without AC. Except at
from each LOD are the ones contributing more to lower very low rates, where the PLTW is still reconstructing up-
the reconstruction error, while bits from negligible coeffi- per LODs and does not benefit from the smoothing effect
cients arrive last. Table 1 shows renderings of the bunny of subdivision (while its competitors do), PLTW always re-
model at different stages of the decoding process. Once the sults in higher PSNRs for the same bit-rate. It is also notice-
base mesh (LOD 0) is received, it is subdivided once and able how none of the SPIHT-based coders is able to reach
LOD 1 is progressively reconstructed. When all coefficients the same PSNR as the PLTW coder even employing 160%
of LOD 1 have been decoded, the mesh is subdivided again (SPIHT-AC) or 330% (MPEG-4) of the bits used by PLTW
and details in LOD 2 are processed. LOD 3 and forthcom- for the same quantisation set of values. The poor results of
ing levels are sequentially decoded until the whole bit-stream the “WaveSurf ” coder are mostly due to the overhead intro-
is read. duced to support view-dependent transmission of coefficient
Figure 2 shows the rate-distortion (PSNR as a function of trees.
the number of bits per vertex) curves obtained for two typ- WSSs permit to code the shape of a 3D model in a mul-
ical 3D models by our PLTW coder, which includes arith- tiresolution manner with very good compression, but require
metic coding (AC) as a final step, and an a version of the a large CPU overhead for a fine-grained, on-the-fly control
SPIHT algorithm with AC. In the case of SPIHT, the bits of the content complexity in execution time regulated ap-
from each LOD have been individually plotted: LODs 1 and plications such as networked, interactive 3D games. Figure 4
2 are quickly reconstructed because their details are the ones shows that the CPU overhead for controlling the execution
contributing the most to lower the reconstruction error. It time with MPEG-4’s “WaveSurf ” tool is sometimes as large
would seem clever to cut the stream or stop decoding after as the 3D graphics rendering execution time itself [10, 11].
some point (e.g., 0.75 b/v for LOD 1 or 1.5 b/v for LOD 2) if Moreover, typical implementations of WSSs multiply by four
those two coarsest LODs are enough, or the only manageable the number of triangles in every subdivision step, which en-
ones, since the bits to come will hardly increase the PSNR. ables only very discrete LOD management, and therefore
However, even in those cases, the decoder needs to build the yields abrupt and often disturbing quality changes while only
whole detail tree to be able to follow the branching of the supporting coarse-grained adaptation to a target execution
SPIHT algorithm. On the contrary, the PLTW decoder is able time. Besides improving the compression efficiency and the
to stop decoding exactly at the desired LOD without allocat- adequacy to weak terminals of “WaveSurf ” with the PLTW
ing extra resources for further LODs—and even with a lower technique, we have introduced some add-ons to enable a
reconstruction error! low-complexity, yet efficient fine-grained quality/execution
Figure 3 plots the rate distortion curves for the PLTW time tradeoff in execution time control, as shown by the up-
coder, the same AC version of SPIHT as above (overall com- per curves of Figures 4 and 6.
4 EURASIP Journal on Advances in Signal Processing

80 70

70 60

60 50
PSNR

PSNR
50 40

40 30

30 20

20 10
0 1 2 3 4 5 6 0 1 2 3 4 5 6 7
Bits/vertex Bits/vertex

PLTW-AC SPIHT-AC LOD3 PLTW-AC SPIHT-AC LOD3

SPIHT-AC LOD1 SPIHT-AC LOD4 SPIHT-AC LOD1 SPIHT-AC LOD4
SPIHT-AC LOD2 SPIHT-AC LOD2
(a) (b)

Figure 2: PLTW versus SPIHT for the Max Planck (a) and bunny (b) models.

80 70

70 60

60 50
PSNR

PSNR

50 40

40 30

30 20

20 10
0 2 4 6 8 10 12 14 0 1 2 3 4 5 6 7 8 9
Bits/vertex Bits/vertex

PLTW PLTW
SPIHT-AC SPIHT-AC
MPEG-4 MPEG-4
(a) (b)

Figure 3: PLTW versus SPIHT and MPEG-4’s “WaveSurf ” for the Max Planck and bunny models.

To achieve this target, the “WaveSurf ” mesh regions are tant or unimportant. Nonuniform subdivision of a WSS does
progressively decoded in a continuous LOD fashion, by sub- not necessarily create all four children of a triangle, but first
dividing only the important regions of the geometry, as checks the importance of every potential child for a specific
shown in Figure 5. The importance and order for subdivid- value of ht before it is added. In this way, additional triangles
ing the triangles is given by their impact on the error to are only introduced (and hence execution time increased)
the target mesh, that is, the triangles that decrease this er- when they really contribute to an improved visual quality of
ror the most are subdivided first. We detect importance with the mesh. Of course, one must then worry about the “cracks”
a heuristic [12] for which values smaller or larger than a that may appear when rendering the mesh, but this problem
threshold parameter ht are, respectively, detected as impor- can be easily solved [7, 13].
Francisco Morán et al. 5

180 Controlled rendering time 250

Target
160
140 200
Predicted time Measured time
120
Time (ms)

Time (ms)
150
100 Uncontrolled
80 rendering time
100 Optimisation time
60
Content adaptation time (floating point) Adaptation time
40
50
20
0 0
1 101 201 301 401 501 601 701 801 901 1001 1 101 201 301 401 501 601 701 801 901 1001
Frame Frame
(a) Software: embedded device
Figure 4: Content adaptation for execution time control.
10
Predicted time
Measured time

Time (ms)
5

Adaptation time

Optimisation time

0 0.1 0.3 0.5 0.7 0.9 1 0

1 501 1001 1501 2001 2501 3001 3501 4001
ht
Frame

Figure 5: Continuous LOD with adaptively subdivided WSSs. (b) DirectX: PC-Geforce Fx5900t

Figure 6: Controlled execution time on two different platforms for

a 3D scene walkthrough with a moving character.

These nonuniformly subdivided WSS meshes allow a 2.2. Combined 2D textures + 3D geometry data
fine-grained control of the resolution of the geometry,
resulting in small variations of the visual quality while In order to choose a coding tool and format for textures, we
achieving a target execution time. With special techniques first carried a comparative study, with respect to the consid-
using LOD-based moving windows [10], the complexity of ered criteria and desired functionalities, between the classical
the subdivision control is largely reduced, resulting in an DCT-based solution, JPEG, and two wavelet-based, and also
overhead of only a small percentage in the final decoding and already standardised solutions: JPEG 2000 [3] and MPEG-4’s
rendering execution time, as shown in Figure 6 for two dif- native tool for still images, VTC (Visual Texture Coding)
ferent terminals: a high-end PC and a low-end CP [11]. [15]. We chose JPEG 2000, which is now supported inside
To steer the execution time control, the execution time, MPEG-4 as a format for textures thanks to a proposal of ours.
and especially the rendering time, should be estimated for a Besides the execution time variation with the platform
large range of triangle budgets. We have used previously re- and content parameters [13], we also observed the linear-
ported performance models for the software [13] and hard- ity of the cost with the object parameters in the bit-rate of
ware [14] rendering pipelines, according to which the most the textured MPEG-4 objects: with a regression coefficient
important parameters are the number V of processed ver- of 93% measured over 60 objects, the original MPEG-4 file
tices (for the vertex processing) and the number F of frag- size s decreases roughly bilinearly with decreasing JPEG 2000
ments (for the rasterisation); additional parameters impor- texture LOD (with negative slope −m1 ) and decreasing ob-
tant for the software model are the number S of spans and ject mesh LOD (with negative slope −m2 ). Figure 7 illustrates
the number T of visible triangles. The coefficients of the per- this trend for two OLGA objects.
formance model are derived with an offline calibration pro- Small file sizes s with large m1 and m2 correspond to
cedure that first measures on the device the rendering time small bit-rates that decrease very rapidly with decreasing
for many different objects with different sizes (F) and com- LOD: those objects representing only a small fraction of the
plexity (V and T), and then computes the average values of total bit-rate at all LOD levels, they have low priority to be
the coefficients cα (α ∈ {T, F, S}) through multilinear regres- scaled for global (over all objects) bit-rate adaptation. On the
sion analysis. A mean error of only 5% between estimated other extreme, large s with small m1 and m2 correspond to
and measured execution time has been observed in the case large bit-rates that decrease very slowly with decreasing LOD,
of the software rendering pipeline [13]. hence representing hardly any chance of downscaling for
6 EURASIP Journal on Advances in Signal Processing

(a) (b)

140000 160000

120000 140000

120000
100000
100000
80000
Size
Size

80000
60000
60000
40000 S7 40000 S7
l

20000

vel
eve

S4 20000 S4

e le
le l

ng l
ng

0 0
a

a
S1 1
Tri

2 S1

Tri
3 2
JPEG level 4 3 4
JPEG level

140000–160000 60000–80000 140000–160000 60000–80000

120000–140000 40000–60000 120000–140000 40000–60000
100000–120000 20000–40000 100000–120000 20000–40000
80000–100000 0–20000 80000–100000 0–20000
(c) m1 = −33395; m2 = −198 (d) m1 = −32103; m2 = −478

Figure 7: Bilinear dependency of MPEG-4 file size on 2D texture and 3D mesh LODs.

global bit-rate adaptation. Consequently, large s with large spatial coordinates but also normals or texture coordinates),
m1 and/or m2 are the most appealing candidates for bit-rate some kind of redundancy in the animation is usually ex-
adaptations; starting from a large full resolution bit-rate con- ploited: either temporal, and then linear or higher-order in-
tribution, they scale very well by adjusting the texture and/or terpolation is used to obtain the value of the desired at-
mesh LOD. tribute between its sampled value at certain key frames, or
Together with the improvements introduced by the spatial, and then nearby vertices are clustered and a unique
PLTW and BBA tools (see below), a global quality versus bit- value or transform is assigned to each cluster. MPEG stan-
rate versus execution time control can be obtained over all dardised an approach for compression of generic interpo-
objects. The precise details of this intelligent global adapta- lated data [18], able to represent coordinates and normal
tion are beyond the scope of this paper, since it is mainly re- interpolation. While generic, this approach does not exploit
lated in finding heuristics for approximately solving an NP- the spatial redundancy. Concerning avatar animation, one of
hard knapsack problem: the interested reader is referred to the most used animation content for games, ISO/IEC pub-
[16, 17] for a framework of 3D interobject adaptation using lished in 1999 [15] and 2000 [19], under the umbrella of
some tabular characteristics of each object. the MPEG-4 specifications, is a set of tools named face and
body animation (FBA) [20] allowing compression at very low
2.3. Animation data bit-rates. The limitations of FBA consist mainly in the rigid
definition of the avatar and the difficulty to set up the pro-
To represent compactly the data required by the animation posed deformation model. Some other methods reported in
of textured 3D models (varying vertex attributes: essentially the literature are quantisation of the motion type [21], data
Francisco Morán et al. 7

Original model QEM-simplified model AC-QEM-simplified model

(1151 vertices) (491 vertices) (497 vertices)
(a) (b) (c)

Figure 8: AC-QEM versus QEM: qualitative results for the dragon model.

200
15
150
10
100 5
50
0
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21
0 −5
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22
−50 −10
A C
B
−100 −15
(a) (b)

Figure 9: Movement along the x(), y(), and z() axes of the centre of a single bone (a) and all extreme bones (b).

transmission scalability by exploiting the 3D scene structure fication procedure. We applied the developed algorithm to
[22], and quantisation to achieve data compression and in- OLGA animated objects, previously converted into MPEG-4
corporate intelligent exploitation of the hierarchical struc- compliant skinned models. Figure 8 shows the comparative
ture of the human skeletal model [23]. results of an animated model simplification for the devel-
At the time the OLGA project started, we were [24] in oped approach, called animation-constrained QEM (AC-
the final stage of standardising BBA, an extension of FBA QEM), versus plain QEM.
within MPEG-4 AFX [2]. BBA allows to represent animated, Decoding and rendering animation data on small mem-
generic 3D objects based on the skin and bones paradigm, ory devices such as CPs require severe server-side compres-
and to transmit the animation data at very low bit-rates by sion. To decrease even more the size of animation data, we
exploiting both the temporal and spatial redundancies of the implemented a frame reduction algorithm: instead of trans-
animation signal. Within OLGA, we addressed the termi- mitting all frames, we have the server transmit some key
nal/network adaptation, compression and rendering of BBA- frames only, and let the decoder guess intermediate frames
based content. We considered the adaptation of animated by linear interpolation (NB: MPEG-4 supports nonuniform
content at two levels: geometry simplification constrained by temporal interpolation by indicating at each key frame the
dynamic behaviour [25] and animation frame reduction. number of intermediate frames to be computed).
The dynamic behaviour was expressed as constraints Given an original animation sequence of n frames, to ob-
used to parameterise the well-known mesh simplification tain a simplified sequence with m frames (m < n) approxi-
quadric error metrics (QEM) technique [26]. We introduced mating best the original curve, the area between the original
a weighting factor to specify how a given set of bones influ- curve and the one reconstructed by linear interpolation must
ences the simplification procedure. The biomechanical char- be minimised. For instance, Figure 9(a), showing the move-
acteristics (i.e., the relationships between skin and bones) ment during 22 frames of the x, y, and z coordinates of the
were directly exploited to constrain and control the simpli- centre of a bone, illustrates how removing frames no. 4 or no.
8 EURASIP Journal on Advances in Signal Processing

19 is less critical, from a distortion viewpoint, than removing Lobby GCS

frames no. 3 or no. 8. server
Considering this condition for all bones, or even for the
ZGS
subset of extreme bones, as shown in Figure 9(b), the optimi-
Game Content
sation problem becomes difficult to solve. To overcome the network delivery LCS
complexity, we adopted an incremental approach. network
···
(i) We first compute for each extreme bone, frame, and
ZGS LCS LCS
coordinate, the area of the triangle defined by the orig-
inal curve and the one reconstructed by linear interpo-
lation. In the case of the bone of Figure 9(a), its x coor- Game ··· ···
··· ···
··· client
dinate for frame no. 15 (marked as B) would be recon-
structed (erroneously) by linear interpolation between Figure 10: Overall network architecture.
its values in frames no. 14 (A) and no. 16 (C), so the
area of triangle ABC is a measure of the error caused
by omitting frame no. 15.
others (and itself, of course). A dynamic procedure governed
(ii) Then, for each frame, we sum these all the error areas by the central lobby server (LS) decides upon creation of each
for all extreme bones and coordinates. The minimum match which client will host its ZGS. The LS monitors the
of the sums indicates the frame that has to be removed. ZGS while the match is going on and, if it detects that the
We iterate the algorithm until only m frames remain, ZGS fails (or disconnects from the game, perhaps because its
and generate a new BBA stream by encoding those m owner simply decides to switch it off), it starts a replacement
frames, indicating for each the number of intermediate ZGS on another PC and transfers all clients to it. In such a
frames to be obtained by linear interpolation on the case, the game state is (mostly) preserved thanks to the back-
terminal. ups that are sent periodically from the ZGS to the LS.
The advantage of this incremental approach where frames (ii) The content delivery network (CDN) is the spe-
are removed one by one is the fine granularity of the file size: cialised subsystem that we developed to enable live 4D con-
in a time-variant environment of the network capabilities, tent update while playing, and to perform dynamic adapta-
it is possible to dynamically adapt the size of the animation tion of that content to terminals in a distributed fashion. It
stream to the changing constraints of the network. is formed by a global content server (GCS), the single point
of upload for new content, and a number of adaptation and
3. CROSS-NETWORK DISTRIBUTED delivery nodes called local content servers (LCSs) that, anal-
CONTENT DELIVERY ogously to the ZGS, are also hosted by game clients.
Both networks, GN and CDN, meet at their edges, as the
3.1. Overall network architecture LS coordinates both and game clients also connect to both,
and it is not unusual that a single client PC be a node in both
The implemented network architecture follows a dual de- networks, since it can host both a ZGS and an LCS. However,
sign. There are two different subnetworks within the system, from a logical point of view, they are different entities.
shown in Figure 10.
(i) The game network (GN) holds the game logic, keep- 3.2. Content delivery network
ing synchronisation among its nodes and therefore enabling
a multiplayer online game. We built it, as a mere support Sending or updating game content (i.e., objects to be ren-
service for our system, with off-the-shelf components, so dered) over the network is not a frequently used option, al-
the implemented game engine can cope with heterogeneous though multiplayer online games pushing content through
clients and networks, using standard protocols (HTTP, XML- the network instead of locally storing all data do exist [29].
RPC, etc.) to help interoperability. However, most of these games reduce a priori the transmis-
We chose a turn-based role-playing game (RPG) as a test sion bandwidth by subdividing the world in subworlds (3D
bed, instead of a faster-paced game genre, to soften the re- tiles) and referencing prestored items, and texture data is sel-
quirements on the GN; nevertheless, we added some basic dom transmitted. We chose instead to enable live update, dis-
tools such as dead reckoning [27, 28] and simple latency tribution, and adaptation of content.
equalisation to ensure that clients had a comparable user ex- This adaptation requires extensive CPU power and mem-
perience. User validation showed indeed that there was no ory. It is not practical to serve dynamically rendered con-
statistically significant difference between PC and CP players tent using a pure client-server architecture, and that is why
with respect to game experience, and network delay did not the peer-to-peer (P2P) model was chosen. Distributing con-
negatively affect the players’ impression, provided it stayed tent across networks is one area in which P2P technologies
within certain bounds. have experienced an enormous boom recently [30]. In our
The GN architecture is distributed in the sense that dif- case, since the main focus of our work was on-the-fly con-
ferent matches are hosted by separate servers, called zone tent adaptation, our system is a hybrid of a content deliv-
game servers (ZGSs). ZGSs are run by client PCs: in every ery network and a distributed computing system for which,
match, one of the clients acts as the game logic server for all instead of a generic and heavyweight approach such as grid
Francisco Morán et al. 9

computing [31], we chose a more specialised distributed P2P The delivery process involves continuous interaction be-
computing model. The background idea comes from sev- tween the client and its two assigned servers: the client inter-
eral large-scale experiments done on Internet computing, the acts with the game world as it is given by the ZGS; whenever
most famous being probably SETI@home [32]. the ZGS delivers indication of a new content element, the
Our target is then to use residual computing power from client contacts its LCS and requests that item, which is then
the client nodes, which must be kept free enough to perform downloaded and rendered. Each request consists of an object
their own individual tasks, notably playing the game. Some identifier and a list of quality parameters. The LCS proceeds
studies have been done on user acceptability of the use of sys- to optimise the locally cached content (or download it, on
tem resources by external processes [33]; our objective has cache misses, from the GCS) according to those parameters
been that the adaptation tasks never exceed a given thresh- by using the developed content adaptation tools, and gener-
old on system load. This is enforced by the load balancing ates a bit-stream encapsulating the optimised content, in a
procedure described below. format suitable for the client. All meshes, animations, and so
The final key features of the CDN are the following. forth, that are used in a game, are retrieved from the LCS.
After the original initialisation, clients do not contact the
(i) The system works through very heterogeneous net-
LS again: all interaction is done through their LCS, including
works and terminals, from high-end 3D graphic PCs
reassignments to another LCS. Given that network delays in
connected to broadband Internet to mobile handsets
reassignments may be significant since LCSs may be topolog-
connecting through 3G networks, all simultaneously
ically separated, job migration is minimised by establishing a
active in the same game, and interacting with each
long-term procedure: clients are assigned an LCS upon en-
other.
tering the network, and they continue using the same LCS
(ii) The LCSs are not passive distribution nodes: on the until they are reassigned. The load balancing procedure, rep-
contrary, they actively adapt the content to the client resented in Figure 11, is as follows.
characteristics before delivery. Adaptation is done
through a set of simplification tools [34], and the LCSs (i) A threshold is put on the maximum CPU load and
cache the result of simplifications to save processing bandwidth of the LCS, depending on the device char-
effort. acteristics and connectivity. Obviously, that threshold
is set well below 100% (load), since the LCS machine is
(iii) A content adaptation server is installed on every client after all mostly a game client, and we do not want the
PC, and may be called upon dynamically by the lobby LCS adaptation processes to hamper the game experi-
server to act as an LCS, depending on system condi- ence.
tions, as explained in Section 3. (ii) The LS polls CPU load and bandwidth usage of the
(iv) The amount of available content in the game is vari- LCS at regular intervals, to monitor its status. To avoid
able, and can be updated from all nodes in the net- exceeding the threshold between polls, prediction over
work. Any game client can create its own content (in the last received data is used so that short-term fu-
standard formats) and insert it into the game in real ture conditions are anticipated. Simple linear predic-
time, by uploading it to the GCS and using the ZGS to tion has been initially chosen, but other schemes are
add a reference to the new content in the game; other also possible [35, 36].
players will download the content from their LCS as (iii) When the threshold is expected to be shortly reached,
needed by game information provided by the ZGS. the LS chooses a replacement from the pool of avail-
able LCSs, and sends a message to the loaded LCS to
Given the P2P properties of the CDN, some scalability is in-
redirect all future adaptation requests to the replace-
herent to it: it is likely that new game clients entering will
ment; game clients change their assigned LCS as they
also bring new servers, thus levelling the capacity of the net-
receive such redirections. If the LCS load later goes be-
work. However, as playing the game makes the clients behave
low a safety value, the LS tells it to start accepting adap-
unpredictably from the point of view of content requests, a
tation tasks again.
means of ensuring dynamic adaptation to changing condi-
tions must be provided. This procedure could be classified as “local decision global
Each LCS serves a number of game clients, following an migration” [37]: the decision to redirect a job is taken based
arrangement that can evolve to accommodate changes in net- only on the state of the affected node (local decision), but
work and client conditions. On initialisation, a game client once decided to be transferred, the job can travel anywhere
connects to the LS and sends a request for joining a game. As in the CDN (global migration).
side information, it also sends its node characteristics, that The load produced by adaptation tasks may depend on
is, computing power and network bandwidth. The LS logs in various factors: the content to adapt, the adaptation parame-
each game client and assigns it a ZGS, in charge of delivering ters (e.g., the simplification level), or the number of concur-
all the game logic to the client, and an LCS, which will deliver rent adaptations. The resulting serve time is, in general, the
all the game content elements to it. In parallel, and depend- sum of the time needed for adaptation plus the time for de-
ing on node characteristics, the LS may tell the client to start livery; if the LCS has already the adapted content, only the
an LCS, which from then on listens to requests for adapted latter time will count; if the LCS does not have the content at
content (it may also tell the client to start an inactive LCS, all, the time needed to fetch it from the GCS must be added
keeping it for future needs). as well to the sum above.
10 EURASIP Journal on Advances in Signal Processing

adaptation time, for a given content file. Since in general it

Lobby
server
GCS is very difficult to know beforehand the pattern of adapta-
tions that an LCS is going to receive (the requested content
nd depends from the behaviour of the client within the game),
ma load prediction and balancing is important to be able to keep
LCSs under control.
om
tc
A similar procedure to that for load balancing is used for
c
ire
Overloaded fault tolerance. In case a client does not receive timely re-
d
Re

Spare
LCS sponses from its LCS, it will contact the LS and ask for a re-
LCS placement. The LS will assess the situation (due to polling it
est
would probably have already detected the problem) and as-
e qu sign the best available LCS to the client.
R
ion We have also investigated a number of variants on the
i rect
d
Re very ··· network architecture to improve efficiency and better adapt
i
Game Del ···
to different needs in content distribution. Some of these al-
client ··· ···
ternatives are the following.
Figure 11: Dynamic load balancing mechanism. (i) Enabling peer cache lookups, through a “hit or noth-
ing” procedure: the peer only delivers the content if it
is in its cache and already adapted as needed; neither
3000 adaptations nor upstream GCS fetches are resolved.
2500 This is similar in concept to the sibling cache process
in the Internet Cache Protocol [38].
Total time (ms)

2000
(ii) 2-level hierarchy: a new level between the GCS and the
1500
LCS, with nodes called content aggregators (CAs), act-
1000 ing as a higher level cache, gathering both unadapted
500 content upstream (from the GCS) and adapted con-
tent downstream (from the LCS). If the LCS receives
0
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 unadapted content, as soon as it finishes the adapta-
Number of simultaneous clients tion it delivers it to both the original requester and the
CA, thus growing the CA cache.
Max
Avg Though the topology of the network increases in complex-
Min ity with the latter versions, the protocol itself remains quite
simple and the procedure followed by each node is straight-
Figure 12: LCS total serve time versus number of clients.
forward, thus ensuring general simplicity in each node.
Finally, game interactivity can be further improved by
enhancing the system with techniques such as automatic
Figure 12 shows the total serve time per client on an coarse adaptation and delivery of items (to create fast pre-
LCS that is busy doing a number of adaptation tasks, as a views), background anticipative prefetching of items as sug-
function of the number of simultaneous adaptations, using gested by the game engine or priority queues in adaptation
a medium-powered machine with a broadband connection. servers.
To eliminate the dependency from the content, all simplifi-
cation tasks were made over the same file (always ensuring 4. CROSS-PLATFORM 3D RENDERING
that adaptation was forced, instead of using an adapted ver-
sion in the cache). As the graph shows, the system scales up 4.1. Software platforms
quite well, since the time to adapt and deliver content is inde-
The OLGA technology supports a variety of terminals, which
pendent from the number of simultaneous adaptations (pro-
were used within the project to test and validate the scalabil-
vided the load does not go beyond a congestion limit). Most
ity of game content. The main focus was on the realm of PCs,
of that time is spent in adaptation: simplification of the con-
ranging from high-end gaming ones to laptops, but also mo-
tent takes on average 85% of the total time, while only 15%
bile terminals were used. Two software platforms were pro-
is spent in delivery (obviously, if the adapted content were in
duced.
cache, the serve time would improve considerably), since we
have favoured lower bandwidth usage. (i) GOAL, our game test bed, is available both on MS
LCS load and bandwidth usage values were also measured Windows-based PCs and on CPs running Symbian
against the number of simultaneous clients being served. OS v8 and supporting J2ME, notably the Nokia 6630.
Variations using different content files show different practi- Game logic was implemented on both versions of the
cal limits in the number of simultaneous clients, but the gen- game, and decoders integrated for the simplified con-
eral shape stays the same. Also, it has been found that there tent downloaded from the network. For the CP, a part
is no significant impact of the simplification level on total of the software is programmed in Java, and the content
Francisco Morán et al. 11

(a) (b) (c)

(d) (e) (f)

Figure 13: Some OLGA models loaded in the PC (top) and CP (bottom) versions of the 3D graphics MPEG-4 player.

decoders are programmed in Symbian, the Symbian ments drive new applications such as real 3D viewing. It has
framework being connected through a socket with the been demonstrated how to optimally design lenticular sheets
Java game engine. Rendering of the final graphics is to turn a flat, 2D matrix display into an auto-stereoscopic
done in software on the ARM embedded in the OMAP multiview display [41]. The resulting displays can present the
processor of the Nokia 6630. For screen shots of GOAL viewer with different images from various slightly different
on both types of terminals, see Figure 1. viewing angles. A format highly suitable for content trans-
(ii) Besides, we have developed a stand-alone MPEG-4 mission for such multiview displays is video enriched with
player to visualise textured 4D content on both PCs depth information [42, 43]. The format allows for minor dis-
and CPs. We selected a small number of the scene placements of foreground objects with respect to background
graph nodes defined in the MPEG-4 Systems specifica- scenery that are needed to present the viewer with the re-
tion [39], which is enough to represent static and ani- quired different images [44].
mated textured 3D objects. Then, starting from GPAC To make the game experience more immersive [45], we
[40], an open source decoder of MPEG-4 BInary For- have endowed some terminals with such autostereoscopic
mat for Scene (BIFS), we derived a simplified BIFS de- 3D displays. Although, in theory, multiple images from dif-
coder by implementing just the selected nodes. To al- ferent viewpoints can be rendered individually on the de-
low texture mapping, we plugged in both JPEG and vice, this solution is not optimal. From both the bandwidth
JPEG 2000 decoders and, to support animation, we op- and computational complexity points of view, it is desir-
timised the initial BBA decoder we had developed for able to render only one viewpoint and provide the depths
PC and also ported it to Symbian OS v8 for the selected of the pixels in the computed view to the display. Subse-
CP (Nokia 6630). Finally, we developed the rendering quently, a dedicated processor in the display can render the
layer by using DirectX 9 for PC and OpenGL ES for CP. desired viewpoints at high quality [44]. In the GOAL ter-
Figure 13 shows some models loaded in the PC and CP minals, we have adopted the latter approach, and provide
versions of the 3D Graphics MPEG-4 player. the depth information that is available in the z-buffer of the
GPU to the 3D display. Therefore, the 3D content is trans-
ferred through the OLGA framework to the device, subse-
4.2. 3D displays quently used to render the scene according to the current
game status. Next, the frame and depth information are
Special attention has given to the final rendering of 3D gam- transferred to the 3D display, which renders the final scene in
ing content. Nowadays, new terminal and display develop- multiview 3D, as suggested by Figure 14—which, of course,
12 EURASIP Journal on Advances in Signal Processing

(a) (b)

Figure 14: Screen shot of GOAL: image and depth buffers (a), and actual image on autostereoscopic 3D display (b).

cannot convey the real 3D experience when printed on 2D Summarising, we have managed to integrate a chain
paper! of content coding, transmission and rendering technolo-
gies into a heterogeneous infrastructure and terminal set,
5. CONCLUSIONS demonstrating real-time interactive 4D content adaptation.
We have developed a distributive multiplayer 4D game as a
Today’s multiplayer 4D games often rely on dedicated/pro- test bed but, more importantly, we have developed a new
prietary technological solutions for their servers (e.g., mas- framework to develop distributive multiplayer 4D games
sively parallel, brute-force grid computing), and scale down (or other multimedia applications with heavy and highly
content a priori, according to the bandwidth or render- variable bandwidth and rendering requirements), and our
ing power of the “weakest” node in the infrastructure. framework hooks to a complete toolkit of standardised con-
The OLGA consortium opted for a completely different tent representation and compression formats (MPEG-4 AFX,
paradigm: thanks to scalable coding of the 3D geometry, tex- JPEG 2000, XML), enabling easy deployment over existing
ture, and animation data, gaming content is automatically infrastructure, while not impeding well-established practices
adapted to heterogeneous platforms and networks, and the in the game development industry.
processing load distributed among the resources available in
a P2P architecture. Indeed, OLGA’s 4D content is not stored REFERENCES
locally on one single server or local storage medium (e.g.,
DVD), but is rather distributed over a multitude of servers [1] T. H. Apperley, “Genre and game studies: toward a critical ap-
spread all over the network with adequate load-balancing proach to video game genres,” Simulation & Gaming, vol. 37,
and fault-tolerance policies, and possibly hosted at the most no. 1, pp. 6–23, 2006.
powerful PCs of the players themselves! [2] ISO/IEC JTC1/SC29/WG11, “Standard 14496-16,” a.k.a.
The 4D content is actively pushed from the available “MPEG-4 Part 16: Animation Framework eXtension (AFX)”,
ISO, 2004.
servers to the gaming terminals but, since new specialised 4D
[3] ISO/IEC JTC1/SC29/WG1, a.k.a. JPEG (Joint Photographic
content compression tools are provided to end users as well
Experts Group): “Standard 15444-1”, a.k.a. “JPEG 2000 Part
as to game designers, the players can publish their own con- 1: Core coding system”, ISO, 2004.
tent, which then becomes part of the persistent world, and [4] XML (eXtensible Markup Language) Core Working Group,
benefits from OLGA’s standardised framework for adapting “XML 1.0 (4th ed.),” W3C (World Wide Web Con-sortium),
scalable content to the varying processing and bandwidth ca- http://www.w3.org/TR/2006/REC-xml-20060816/, 2006.
pacities of a heterogeneous infrastructure, and to the very [5] A. Said and W. A. Pearlman, “A new, fast, and efficient im-
different rendering power of heterogeneous terminals. These age codec based on set partitioning in hierarchical trees,”
4D content compression tools do not impose constraints on IEEE Transactions on Circuits and Systems for Video Technol-
content complexity: game developers and players are free in ogy, vol. 6, no. 3, pp. 243–250, 1996.
their creativity, and the tools adapt to any circumstances— [6] A. Khodakovsky, P. Schröder, and W. Sweldens, “Progressive
not the other way around, as is usually the case. geometry compression,” in Proceedings of the 27th Annual
Francisco Morán et al. 13

Conference on Computer Graphics and Interactive Techniques [21] M. Endo, T. Yasuda, and S. Yokoi, “A distributed multiuser vir-
(SIGGRAPH ’00), pp. 271–278, ACM, New Orleans, La, USA, tual space system,” IEEE Computer Graphics and Applications,
July 2000. vol. 23, no. 1, pp. 50–57, 2003.
[7] F. Morán, Modelado jerárquico de objetos 3D con superficies de [22] T. Hijiri, K. Nishitani, T. Cornish, T. Naka, and S. Asahara,
subdivisión, Ph.D. thesis, Universidad Politécnica de Madrid, “Spatial hierarchical compression method for 3D streaming
Madrid, Spain, 2001, http://www.gti.ssr.upm.es./ fmb/pub/ animation,” in Proceedings of the 5th Symposium on Virtual
PhD.pdf.gz. Reality Modeling Language (Web3D-VRML ’00), pp. 95–101,
[8] F. Morán and N. Garcı́a, “Comparison of wavelet-based three- ACM, Monterey, Calif, USA, February 2000.
dimensional model coding techniques,” IEEE Transactions on [23] S. Chattopadhyay, S. M. Bhandarkar, and K. Li, “Virtual peo-
Circuits and Systems for Video Technology, vol. 14, no. 7, pp. ple & scalable worlds: efficient compression and delivery of
937–949, 2004. stored motion data for avatar animation in resource con-
[9] M. Avilés, F. Morán, and N. Garcı́a, “Progressive lower trees of strained devices,” in Proceedings of the ACM Symposium on
wavelet coefficients: efficient spatial and SNR scalable coding Virtual Reality Software and Technology (VRST ’05), pp. 235–
of 3D models,” in Proceedings of the 6th Pacific Rim Conference 243, ACM, Monterey, Calif, USA, November 2005.
on Multimedia (PCM ’05), vol. 3767 of Lecture Notes in Com- [24] M. Preda and F. Prêteux, “Virtual character within MPEG-4
puter Science, pp. 61–72, Springer, Jeju Island, Korea, Novem- animation framework eXtension,” IEEE Transactions on Cir-
ber 2005. cuits and Systems for Video Technology, vol. 14, no. 7, pp. 975–
[10] N. Tack, G. Lafruit, F. Catthoor, and R. Lauwereins, “Elimi- 988, 2004.
nating CPU overhead for on-the-fly content adaptation with
MPEG-4 wavelet subdivision surfaces,” IEEE Transactions on [25] M. Preda, S. Tran, and F. Prêteux, “Adaptation of quadric met-
ric simplification to MPEG-4 animated object,” in Proceed-
Consumer Electronics, vol. 52, no. 2, pp. 559–565, 2006.
ings of the 6th Pacific Rim Conference on Multimedia (PCM
[11] K. Tack, G. Lafruit, F. Catthoor, and R. Lauwereins, “Platform
’05), vol. 3767 of Lecture Notes in Computer Science, pp. 49–
independent optimisation of multi-resolution 3D content to
60, Springer, Jeju Island, Korea, November 2005.
enable universal media access,” The Visual Computer, vol. 22,
no. 8, pp. 577–590, 2006. [26] M. Garland and P. S. Heckbert, “Surface simplification using
[12] A. W. F. Lee, W. Sweldens, P. Schröder, L. Cowsar, and D. quadric error metrics,” in Proceedings of the 24th Annual Con-
Dobkin, “MAPS: multiresolution adaptive parameterization ference on Computer Graphics and Interactive Techniques (SIG-
of surfaces,” in Proceedings of the 25th Annual Conference on GRAPH ’97), pp. 209–216, ACM, Los Angeles, Calif, USA, Au-
Computer Graphics and Interactive Techniques (SIGGRAPH gust 1997.
’98), pp. 95–104, ACM, Orlando, Fla, USA, July 1998. [27] IEEE, “Standard 1278.1a for Distributed Interactive
[13] N. Tack, F. Morán, G. Lafruit, and R. Lauwereins, “3D graphics Simulation—Application Protocols,” IEEE, 1998.
rendering time modeling and control for mobile terminals,” [28] J. S. Dahmann, R. M. Fujimoto, and R. M. Weatherly, “De-
in Proceedings of the 9th International Conference on 3D Web partment of defense high level architecture,” in Proceedings of
Technology, pp. 109–117, ACM, Monterey, Calif, USA, April the 29th Conference on Winter Simulation, pp. 142–149, ACM,
2004. Atlanta, Ga, USA, December 1997.
[14] M. Wimmer and P. Wonka, “Rendering time estimation for [29] J. Smed, T. Kaukoranta, and H. Hakonen, “Aspects of network-
real-time rendering,” in Proceedings of the 14th Eurographics ing in multiplayer computer games,” Electronic Library, vol. 20,
workshop on Rendering, pp. 118–129, Leuven, Belgium, June no. 2, pp. 87–97, 2002.
2003. [30] S. Androutsellis-Theotokis and D. Spinellis, “A survey of peer-
[15] ISO/IEC JTC1/SC29/WG11, “Standard 14496-2,” a.k.a. to-peer content distribution technologies,” ACM Computing
“MPEG-4 Part 2: Visual”, ISO, 1999. Surveys, vol. 36, no. 4, pp. 335–371, 2004.
[16] J. Bormans, N. P. Ngoc, G. Deconinck, and G. Lafruit, “Ter- [31] I. Foster, C. Kesselman, and S. Tuecke, “The anatomy of the
minal QoS: advanced resource management for cost-effective
grid: enabling scalable virtual organizations,” International
multimedia appliances in dynamic contexts,” in Ambient In- Journal of High Performance Computing Applications, vol. 15,
telligence: Impact on Embedded System Design, pp. 183–201, no. 3, pp. 200–222, 2001.
Kluwer Academic Publishers, Norwell, Mass, USA, 2003.
[32] D. P. Anderson, J. Cobb, E. Korpela, M. Lebofsky, and D.
[17] P. N. Ngoc, G. Lafruit, J. Y. Mignolet, S. Vernalde, G. Deconick,
Werthimer, “SETI@home: an experiment in public-resource
and R. Lauwereins, “A framework for mapping scalable net-
computing,” Communications of the ACM, vol. 45, no. 11, pp.
worked applications on run-time reconfigurable platforms,”
56–61, 2002.
in Proceedings of International Conference on Multimedia and
Expo (ICME ’03), vol. 1, pp. 469–472, Baltimore, Md, USA, [33] A. Gupta, B. Lin, and P. A. Dinda, “Measuring and un-
July 2003. derstanding user comfort with resource borrowing,” in Pro-
[18] E. S. Jang, J. D. K. Kim, S. Y. Jung, M.-J. Han, S. O. Woo, and ceedings of the 13th IEEE International Symposium on High
S.-J. Lee, “Interpolator data compression for MPEG-4 anima- Performance Distributed Computing, pp. 214–224, Honolulu,
tion,” IEEE Transactions on Circuits and Systems for Video Tech- Hawaii, USA, June 2004.
nology, vol. 14, no. 7, pp. 989–1008, 2004. [34] F. Morán, M. Preda, G. Lafruit, P. Villegas, and R.-P. Berretty,
[19] ISO/IEC JTC1/SC29/WG11, “Standard 14496-2/AMD1,” “A content adaptation approach for on-line gaming on var-
a.k.a. “MPEG-4 Part 2: Visual, Amendment 1: Visual exten- ious networks and terminals,” in Proceedings of International
sions,” ISO, 2000. Digital Games Conference (iDiG ’06), pp. 233–236, Portalegre,
[20] M. Preda and F. Prêteux, “Critic review on MPEG-4 face and Portugal, September 2006.
body animation,” in Proceedings of the International Conference [35] P. A. Dinda and D. R. O’Hallaron, “Host load prediction using
on Image Processing (ICIP ’02), vol. 3, pp. 505–508, Rochester, linear models,” Cluster Computing, vol. 3, no. 4, pp. 265–280,
NY, USA, September 2002. 2000.
14 EURASIP Journal on Advances in Signal Processing

[36] R. Wolski, N. T. Spring, and J. Hayes, “The network weather Marius Preda is Associate Professor at the
service: a distributed resource performance forecasting service ARTEMIS Department of the GET INT
for metacomputing,” Future Generation Computer Systems, (Groupe des Ecoles des Télécommunica-
vol. 15, no. 5-6, pp. 757–768, 1999. tions-Institut National des Télécommuni-
[37] R. Lüling, B. Monien, and F. Ramme, “Load balancing in large cations) in Evry, France. He received an En-
networks: a comparative study,” in Proceedings of the 3rd IEEE gineering degree in Electronics from Uni-
Symposium on Parallel and Distributed Processing, pp. 686–689, versity Politehnica of Bucharest (Romania)
Dallas, Tex, USA, December 1991. in 1998, and a Ph.D. degree in mathemat-
ics and informatics from Université Paris V-
[38] D. Wessels and K. Claffy, “RFC 2187: Application of Internet
René Descartes (France) in 2001. He started
Cache Protocol (ICP), version 2,” IETF, 1997.
his carrier in Bucharest as a production engineer at Electronica
[39] ISO/IEC JTC1/SC29/WG11, a.k.a. MPEG (Moving Picture Ex- Aplicata and then as a researcher at University Electronica si Tele-
perts Group): “Standard 14496-1”, a.k.a. “MPEG-4 Part 1: Sys- comunicatii. During his Ph.D. studies, he was R&D Engineer at
tems”, ISO, 1999. GET INT ARTEMIS, where he held an R&D Project Manager posi-
[40] J. Le Feuvre, “GPAC,” http://gpac.sourceforge.net./. tion in 2003-2004 before becoming an Associate Professor in 2005.
[41] C. van Berkel and J. A. Clarke, “Characterization and opti- His interests include 3D graphics representation and compression,
mization of 3D-LCD module design,” in Stereoscopic Displays multimedia composition, interactive applications, and multime-
and Virtual Reality Systems IV, vol. 3012 of Proceedings of SPIE, dia standardization. He is actively involved in the management
pp. 179–186, San Jose, Calif, USA, February 1997. of various projects at the national and European levels. He is the
[42] A. Redert, M. O. de Beeck, C. Fehn, et al., “ATTEST: ad- Chairman of the 3D Graphics subgroup of Moving Picture Ex-
vanced three-dimensional television system technologies,” in perts Group (MPEG) and editor of the standards ISO/IEC 14496
Proceedings of 1st International Symposium on 3D Data Process- 16, Animation Framework eXtension (AFX) and ISO/IEC 14496
ing Visualization and Transmission (3DPVT ’02), pp. 313–319, 25, 3DGCM (3D Graphics Compression Model). He is the initia-
Padova, Italy, June 2002. tor of MPEG’s 3DGCM project, aiming at a generic 3D graphics
architecture model allowing to apply MPEG-4 compression on any
[43] C. Fehn, P. Kauff, M. O. de Beeck, et al., “An evolutionary and
scene description formalism.
optimised approach on 3D-TV,” in Proceedings of International
Broadcast Conference (IBC ’02), pp. 357–365, Amsterdam, The Gauthier Lafruit was a Research Scientist
Netherlands, September 2002. with the Belgian National Foundation for
[44] R.-P. Berretty, F. J. Peters, and G. T. G. Volleberg, “Real time Scientific Research from 1989 to 1994, being
rendering for multiview autostereoscopic displays,” in Stereo- mainly active in the area of NMR image ac-
scopic Displays and Virtual Reality Systems XIII, vol. 6055 of quisition, wavelet image compression, and
Proceedings of SPIE, pp. 208–219, San Jose, Calif, USA, January VLSI implementations for image transmis-
2006. sion. From 1995, he was a Research Assis-
tant at the Department of Electronics with
[45] R. Rajae-Joordens, E. Langendijk, P. Wilinski, and I. Heynder-
the Vrije Universiteit Brussel, Belgium. In
ickx, “Added value of a multi-view auto-stereoscopic 3D dis-
1996, he joined IMEC, where he was first
play in gaming applications,” in Proceedings of the 12th Inter-
involved as a Senior Scientist in projects for the low-power VLSI
national Display Workshops in Conjunction with Asia Display
implementation of combined JPEG/wavelet compression engines
(IDW/AD ’05), pp. 1731–1734, Takamatsu, Japan, December
for the European Space Agency. His main current activities are
2005.
with progressive transmission in still image and video coding, scal-
ability in textured 3D models rendering, and resource monitor-
ing in virtual reality. In this role, he has made decisive contri-
Francisco Morán received the degrees of butions to the standardisation of 3D implementation complexity
Telecommunication Engineering and Ph.D. management in MPEG-4. His scientific activities are evolving to-
in telecommunication, both from the UPM wards multicamera video coding, transmission, stereo matching,
(Universidad Politécnica de Madrid, Spain), and visual viewpoint interpolation, with as central interest point
in 1992 and 2001, respectively. Since 1992, the well-balanced tradeoff between algorithmic, architectural, and
he is a Researcher at UPM’s GTI (Grupo de compiler efficiency. The main target is the development of an over-
Tratamiento de Imágenes-Image Processing all optimised system, both end-to-end, as from the perspective of
Group). In 1997, he became a member of design effort and run-time efficiency. He is (co)author of over one
the faculty of UPM’s Department of Sig- hundred scientific publications, four patents, and a couple of book
nals, Systems, and Communications, where chapters and science vulgarization articles.
he is, since 2002, a full time Associate Professor in the knowl-
edge area of Signal Theory and Communications. His research in- Paulo Villegas holds a Telecommunica-
terests include modelling, coding, transmission, and visualisation tions Engineer degree from Universidad
of 3D objects. He has been and is actively involved in projects Politécnica de Madrid. He has been work-
funded by the RACE, ACTS, and IST Programmes of the Euro- ing at Telefónica I + D since 1992, first
pean Commission, for example, OLGA, a STREP from FP6 that in Madrid and, since 2000, in Valladolid,
ended successfully in 2006. He also participates actively since 1996 Spain, in different research groups deal-
in the standardisation activities from ISO’s moving picture ex- ing with multimedia processing, especially
perts group (MPEG). He is (co-)editor of several standards, am- video, successively as a Research Engi-
mendments, and corrigenda related to MPEG-4 (formally, ISO/IEC neer, Project Leader, and Division Manager.
14496), and he is the Head of the Spanish Delegation of MPEG He is now working as a senior research
since 2006. consultant. His main interests are related to digital image and video
Francisco Morán et al. 15

processing, especially in the fields of image and video represen-

tation, content analysis for multimedia libraries, cataloguing and
searching applications, multimedia and semantics, and metadata
management and distribution. He has taken part in several Euro-
pean research projects, and is also involved in international stan-
dardisation initiatives, especially those of the MPEG committee.

Robert-Paul M. Berretty received a Ph.D.

degree in computing sciences from Utrecht
University (The Netherlands) in 2000. In
2001 he was a Postdoctoral Researcher at
the University of North Carolina at Chapel
Hill (USA). In 2002, he joined Royal Philips
Electronics. There, he started as a Senior
Scientist. Currently, he is cluster leader of
the video processing cluster of the Digital
Signal Processing group. He was coordina-
tor of the OLGA project. His main interests include applied com-
putational geometry, computer graphics and image processing, as
well as object modelling in both still images and video streams. He
is currently working on signal processing for 3D display applica-
tions.